How to lazy allocate zeroed memory?

Asked
Active3 hr before
Viewed126 times

7 Answers

90%

If you're trying to emulate calloc with malloc (i.e. Use malloc but receive zeroed memory) then you can do so with memset:,There are a few mechanisms to get pre-zeroed memory from the operating system:,The POSIX shared memory segments can also zero,mmap(2)'s MAP_ANONYMOUS flag forces the contents to be initialized to zero.

The memory comes pre-zeroed:

   This volume of IEEE Std 1003.1 - 2001 specifies that memory
   objects have initial contents of zero when created.This is
   consistent with current behavior
   for both files and newly
   allocated memory.For those implementations that use physical
   memory, it would be possible that such implementations could
   simply use available memory and give it to the process
   uninitialized.This, however, is not consistent with standard
   behavior
   for the uninitialized data area, the stack, and of
      course, files.Finally, it is highly desirable to set the
   allocated memory to zero
   for security reasons.Thus,
      initializing memory objects to zero is required.

It appears that this memory is zeroed at use: mm/shmem.c function shmem_zero_setup():

/**
 * shmem_zero_setup - setup a shared anonymous mapping
 * @vma: the vma to be mmapped is prepared by do_mmap_pgoff
 */
int shmem_zero_setup(struct vm_area_struct * vma) {
   struct file * file;
   loff_t size = vma - > vm_end - vma - > vm_start;

   file = shmem_file_setup("dev/zero", size, vma - > vm_flags);
   if (IS_ERR(file))
      return PTR_ERR(file);

   if (vma - > vm_file)
      fput(vma - > vm_file);
   vma - > vm_file = file;
   vma - > vm_ops = & shmem_vm_ops;
   return 0;
}
load more v
88%

By the way, this is not a fully correct implementation. See the challenges below for a list of problems we're aware of. , If all goes well, your lazy allocation code should result in echo hi working. You should get at least one page fault (and thus lazy allocation) in the shell, and perhaps two. , Try to guess what the result of this modification will be: what will break? , Hint: you'll need to call mappages(). In order to do this you'll need to delete the static in the declaration of mappages() in vm.c, and you'll need to declare mappages() in trap.c. Add this declaration to trap.c before any call to mappages():

Make this modification, boot xv6, and type echo hi to the shell. You should see something like this:

init: starting sh
$ echo hi
pid 3 sh: trap 14 err 6 on cpu 0 eip 0x12f1 addr 0x4004--kill proc
$
load more v
72%

The largest possible memory block malloc can allocate depends on the host system, particularly the size of physical memory and the operating system implementation. ,The C library implementations shipping with various operating systems and compilers may come with alternatives and extensions to the standard malloc interface. Notable among these is: ,If the type of the pointer is changed at its declaration, one may also need to change all lines where malloc is called and cast.,Many different implementations of the actual memory allocation mechanism, used by malloc, are available. Their performance varies in both execution time and required memory.

int array[10];
load more v
65%

Your process eventually exits, and the kernel doesn't need to reclaim the RAM because it was never allocated in the first place.,The kernel finds 256 MiB of unused RAM and gives it to your process by modifying the page table.,There are four main parts here: your program, the standard library, the kernel, and the page tables. You already know your program, so...,So you can see, memset() is very fast and you're not really going to get anything better for large blocks of memory.

The calloc() function is not using some special memory-aligned version of memset(), and that wouldn't make it much faster anyway. Most memset() implementations for modern processors look kind of like this:

function memset(dest, c, len)
// one byte at a time, until the dest is aligned...
while (len > 0 && ((unsigned int) dest & 15))
   *
   dest++ = c
len -= 1
// now write big chunks at a time (processor-specific)...
// block size might not be 16, it's just pseudocode
while (len >= 16)
   // some optimized vector code goes here
   // glibc uses SSE2 when available
   dest += 16
len -= 16
// the end is not aligned, so one byte at a time
while (len > 0)
   *
   dest++ = c
len -= 1
75%

Listing 1  Lazy allocation of memory through an accessor,Note: The term zone is synonymous with the terms heap, pool, and arena in terms of memory allocation using the malloc routines. ,For large allocations, you may also find that it makes sense to allocate virtual memory directly using vm_allocate, rather than using malloc. The example in Listing 2 shows how to use the vm_allocate function.,Batch allocation of memory blocks is supported in OS X version 10.3 and later and in iOS. For information, see the /usr/include/malloc/malloc.h header file.

MyGlobalInfo * GetGlobalBuffer()
load more v
40%

On a page fault on these not allocated address , the kernel allocates new pa and map into the page table.,Kill a process if it page-faults on a virtual memory address higher than any allocated with sbrk().,Handle the case in which a process passes a valid address from sbrk() to a system call such as read or write, but the memory for that address has not yet been allocated.,For the system call read and write , will happen “ page fault ” in walkaddr ,but not go to the usertrap,we need to handle it in walkaddr

12345678910111213141516171819202122232425
load more v
22%

The calloc, malloc, and realloc functions accept zero as an argument. No actual memory is allocated, but a valid pointer is returned and the memory block can be modified later by realloc.,ANSI 4.10.3 The behavior of the calloc, malloc, or realloc function if the size requested is zero,Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.,Feedback will be sent to Microsoft: By pressing the submit button, your feedback will be used to improve Microsoft products and services. Privacy policy.

ANSI 4.10.3 The behavior of the calloc, malloc, or realloc function if the size requested is zero

calloc

ANSI 4.10.3 The behavior of the calloc, malloc, or realloc function if the size requested is zero

malloc

ANSI 4.10.3 The behavior of the calloc, malloc, or realloc function if the size requested is zero

realloc
load more v