How to dynamically load often re-generated c code quickly?

Asked
Active3 hr before
Viewed126 times

7 Answers

dynamically
90%

You coul use many other different approaches, e.g. have a bytecode interpreter and generate for that bytecode, use Common Lisp (e.g. SBCL on Linux which compiles dynamically to machine code), LuaJit, Java, MetaOcaml etc., But TCC does have a weakness: while its compilation time is good, the generated machine code is very slow. (TCC compiles very quickly C code into slow machine code). If the C code is generated, you might follow some different route.... – Basile Starynkevitch Sep 7 '12 at 18:28 ,For a more robust but non-C-based solution, you should probably check out the LLVM project, which does much the same thing but from the perspective of producing JITs. You don't get to go via C, instead using a kind of abstract portable machine code, but the generated code is loads faster and it's under more active development.,You probably won't bother to dlclose a dynamically loaded shared object, except at application clean-up and exit time (but I don't bother at all to dlclose). If you do dlclose some dynamically loaded generated*.so file, be sure that nothing is used in it: no pointers, not even return addresses in call frames, are existing to it.

Second, the main reason to generate C code is to take advantage of a good optimizing compiler (another reason is the portability and ubiquity of C). If you don't care about performance of the generated code (and TCC compiles very quickly C into a very naive and slow machine code) you could use some other approaches, e.g. using some JIT libraries like Gnu lightning (very quick generation of slow machine code), Gnu Libjit or ASMJIT (generated machine code is a bit better), LLVM or GCCJIT (good machine code generated, but generation time comparable to a compiler).

So if you generate C code and want it to run quickly, the compilation time of the C code is not negligible (since you probably would fork a gcc -O -fPIC -shared command to make some shared object foo.so out of your generated foo.c). By experience, generating C code takes much less time than compiling it (with gcc -O). In MELT, the generation of C code is more than 10x faster than its compilation by GCC (and usually 30x faster). But the optimizations done by a C compiler are worth it.

gcc - O - fPIC - shared

Second, the main reason to generate C code is to take advantage of a good optimizing compiler (another reason is the portability and ubiquity of C). If you don't care about performance of the generated code (and TCC compiles very quickly C into a very naive and slow machine code) you could use some other approaches, e.g. using some JIT libraries like Gnu lightning (very quick generation of slow machine code), Gnu Libjit or ASMJIT (generated machine code is a bit better), LLVM or GCCJIT (good machine code generated, but generation time comparable to a compiler).

So if you generate C code and want it to run quickly, the compilation time of the C code is not negligible (since you probably would fork a gcc -O -fPIC -shared command to make some shared object foo.so out of your generated foo.c). By experience, generating C code takes much less time than compiling it (with gcc -O). In MELT, the generation of C code is more than 10x faster than its compilation by GCC (and usually 30x faster). But the optimizations done by a C compiler are worth it.

foo.so

Second, the main reason to generate C code is to take advantage of a good optimizing compiler (another reason is the portability and ubiquity of C). If you don't care about performance of the generated code (and TCC compiles very quickly C into a very naive and slow machine code) you could use some other approaches, e.g. using some JIT libraries like Gnu lightning (very quick generation of slow machine code), Gnu Libjit or ASMJIT (generated machine code is a bit better), LLVM or GCCJIT (good machine code generated, but generation time comparable to a compiler).

So if you generate C code and want it to run quickly, the compilation time of the C code is not negligible (since you probably would fork a gcc -O -fPIC -shared command to make some shared object foo.so out of your generated foo.c). By experience, generating C code takes much less time than compiling it (with gcc -O). In MELT, the generation of C code is more than 10x faster than its compilation by GCC (and usually 30x faster). But the optimizations done by a C compiler are worth it.

foo.c

Second, the main reason to generate C code is to take advantage of a good optimizing compiler (another reason is the portability and ubiquity of C). If you don't care about performance of the generated code (and TCC compiles very quickly C into a very naive and slow machine code) you could use some other approaches, e.g. using some JIT libraries like Gnu lightning (very quick generation of slow machine code), Gnu Libjit or ASMJIT (generated machine code is a bit better), LLVM or GCCJIT (good machine code generated, but generation time comparable to a compiler).

So if you generate C code and want it to run quickly, the compilation time of the C code is not negligible (since you probably would fork a gcc -O -fPIC -shared command to make some shared object foo.so out of your generated foo.c). By experience, generating C code takes much less time than compiling it (with gcc -O). In MELT, the generation of C code is more than 10x faster than its compilation by GCC (and usually 30x faster). But the optimizations done by a C compiler are worth it.

gcc - O
load more v
88%

Memory allocated "on the fly" during run time,dynamically allocated space usually placed in a program segment known as the heap or the free store,It is the programmer's job to deallocate dynamically created space,So once the space has been dynamically allocated, how do we use it?

type
 new int; // dynamically allocates an int 
 new double; // dynamically allocates a double
load more v
72%

Since C is a structured language, it has some fixed rules for programming. One of them includes changing the size of an array. An array is a collection of items stored at contiguous memory locations.  ,How to deallocate memory without using free() in C?,ptr = (float*) calloc(25, sizeof(float));This statement allocates contiguous space in memory for 25 elements each with the size of the float. ,Dynamic Memory Allocation

The “malloc” or “memory allocation” method in C is used to dynamically allocate a single large block of memory with the specified size. It returns a pointer of type void which can be cast into a pointer of any form. It doesn’t Iniatialize memory at execution time so that it has initializes each block with the default garbage value initially. 
Syntax: 
 

ptr = (cast - type * ) malloc(byte - size)
For Example:
Output:
Enter number of elements: 5
Memory successfully allocated using malloc.
The elements of the array are: 1, 2, 3, 4, 5,
  1. “calloc” or “contiguous allocation” method in C is used to dynamically allocate the specified number of blocks of memory of the specified type. it is very much similar to malloc() but has two different points and these are:
  2. It initializes each block with a default value ‘0’.
  3. It has two parameters or arguments as compare to malloc().
    Syntax: 
     
ptr = (cast - type * ) calloc(n, element - size);
here, n is the no.of elements and element - size is the size of each element.
Output:
Enter number of elements: 5
Memory successfully allocated using calloc.
The elements of the array are: 1, 2, 3, 4, 5,

“free” method in C is used to dynamically de-allocate the memory. The memory allocated using functions malloc() and calloc() is not de-allocated on their own. Hence the free() method is used, whenever the dynamic memory allocation takes place. It helps to reduce wastage of memory by freeing it.
Syntax: 
 

free(ptr);
Output:
Enter number of elements: 5
Memory successfully allocated using malloc.
Malloc Memory successfully freed.

Memory successfully allocated using calloc.
Calloc Memory successfully freed.

“realloc” or “re-allocation” method in C is used to dynamically change the memory allocation of a previously allocated memory. In other words, if the memory previously allocated with the help of malloc or calloc is insufficient, realloc can be used to dynamically re-allocate memory. re-allocation of memory maintains the already present value and new blocks will be initialized with the default garbage value.
Syntax: 
 

ptr = realloc(ptr, newSize);

where ptr is reallocated with new size 'newSize'.
Output:
Enter number of elements: 5
Memory successfully allocated using calloc.
The elements of the array are: 1, 2, 3, 4, 5,

   Enter the new size of the array: 10
Memory successfully re - allocated using realloc.
The elements of the array are: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
load more v
65%

For an example of a program that uses the 'dl' interface, try looking at our dynamic-shared library example directory. , For an example of program that uses a static library, try looking at our static library example directory. , Thus, the set of commands we will use to create a shared library, would be something like this: , In order to open and load the shared library, one should use the dlopen() function. It is used this way:


setenv LD_LIBRARY_PATH / full / path / to / library / directory
load more v
75%

Behind the scenes, “Build and reload” is doing a lot of work for you. They:,Click Build & Reload in the build pane, or press Ctrl/Cmd + Shift + B. You can continue to use the standard devtools::load_all() process but it is more risky. Because you’re loading and unloading C code, the chances of corrupting memory are high, and you’re better off with the safer, but slower, “Build & Reload” which installs the package then restarts R.,Use clang instead of gcc to compile your C code: it gives much better error messages. You can make clang the default by creating a ~/.R/Makevars that contains:, Use clang instead of gcc to compile your C code: it gives much better error messages. You can make clang the default by creating a ~/.R/Makevars that contains: CC=clang

usethis::use_rcpp()
load more v
40%

Replace the contents of the header file with this code:,In the editor, replace the contents of the MathLibrary.cpp file with the following code:,Create a DLL project in Visual Studio.,Create a DLL project in Visual Studio.

Replace the contents of the header file with this code:

// MathLibrary.h - Contains declarations of math functions
#pragma once

#ifdef MATHLIBRARY_EXPORTS
#define MATHLIBRARY_API __declspec(dllexport)
#else
#define MATHLIBRARY_API __declspec(dllimport)
#endif

// The Fibonacci recurrence relation describes a sequence F
// where F(n) is { n = 0, a
//               { n = 1, b
//               { n > 1, F(n-2) + F(n-1)
// for some initial integral values a and b.
// If the sequence is initialized F(0) = 1, F(1) = 1,
// then this relation produces the well-known Fibonacci
// sequence: 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

// Initialize a Fibonacci relation sequence
// such that F(0) = a, F(1) = b.
// This function must be called before any other function.
extern "C"
MATHLIBRARY_API void fibonacci_init(
   const unsigned long long a,
      const unsigned long long b);

// Produce the next value in the sequence.
// Returns true on success and updates current value and index;
// false on overflow, leaves current value and index unchanged.
extern "C"
MATHLIBRARY_API bool fibonacci_next();

// Get the current value in the sequence.
extern "C"
MATHLIBRARY_API unsigned long long fibonacci_current();

// Get the position of the current value in the sequence.
extern "C"
MATHLIBRARY_API unsigned fibonacci_index();
load more v
22%

No matter how much we try, it is very difficult to free all dynamically allocated memory. Even if we can do that, it is often not safe from exceptions. Let us look at a simple example:,C++ is a general purpose programming language that offers great flexibility when it comes to system programming and application development. The language, however, comes with some pitfalls requiring attention from developers to ensure quality programming. This article explains how ten of the most common C++ developer mistakes can be avoided.,To avoid such C++ issues, a safer way of reusing of a class/template from the standard library is to use private inheritance or composition.,No matter what happens, after creating the “a” object it will be deleted as soon as the program execution exits from the scope.

No matter how much we try, it is very difficult to free all dynamically allocated memory. Even if we can do that, it is often not safe from exceptions. Let us look at a simple example:

void SomeMethod() {
   ClassA * a = new ClassA;
   SomeOtherMethod(); // it can throw an exception
   delete a;
}
load more v

Other "dynamically-undefined" queries related to "How to dynamically load often re-generated c code quickly?"