The heap is a shared memory resource, and must be managed externally to your process (by the memory manager). The stack, on the other hand, is managed by your process, and any variables pushed onto the stack within your method/function are automatically released/popped when your method/function completes. this is clean, virtually free, and virtually fool-proof.
Malloc
How malloc works?
What can we do with returned address ?
Current chunk (0x10) = actual memory requested (0x08) + bytes to store actual memory size (0x04) + bytes unused (0x04).
There are many ways this can be implemented. Most implementations of C memory allocation functions will store accounting information for each block, either in-line or separately.
One typical way (in-line) is to actually allocate both a header and the memory you asked for, padded out to some minimum size. So for example, if you asked for 20 bytes, the system may allocate a 48-byte block:
- 16-byte header containing size, special marker, checksum, pointers to next/previous block and so on.
- 32 bytes data area (your 20 bytes padded out to a multiple of 16).
free will simply take the address you give it and, assuming you haven't stuffed up that address or the memory around it, check the accounting information immediately before it. Graphically, that would be along the lines of: __ The allocated block __
/ \
+---------+---------------------+
| Header | Your data area ... |
+---------+---------------------+
^
|
+-- The address you are given
Keep in mind the size of the header and the padding are totally implementation defined (actually, the entire thing is implementation-defined (a) but the in-line accounting option is a common one).
The checksums and special markers that exist in the accounting information are often the cause of errors like "Memory arena corrupted" or "Double free" if you overwrite them or free them twice.
The padding (to make allocation more efficient) is why you can sometimes write a little bit beyond the end of your requested space without causing problems (still, don't do that, it's undefined behaviour and, just because it works sometimes, doesn't mean it's okay to do it).
DLMalloc (Named after Doug Lea)
C++ Memory Allocation & Deallocation
While malloc() will work in C++, there is a growing use of the new() function instead. Instead of malloc, we could create a login name with the new function:
char *login = new char[50];
Then, when done, you use delete instead of free:
delete [] login;
Delete is a way of compiler says that the login is now off limits and you can't access.
When a variable got memory from heap using new keyword, the variable is a
raw pointer.
Problem 1:
#include <iostream> #include <vector> using namespace std; int main() { int *vec = new int; delete []vec; return 0; }
Output:
No errors or program output.
Problem 2:
#include <iostream> #include <vector> using namespace std; int main() { vector<int> *vec = new vector<int>; delete []vec; return 0; }
Segmentation fault |
Other examples:
Eg 1:
#include <iostream> #include <vector> using namespace std; int main() { vector<int> vec = new vector<int>; return 0; }
main.cpp: In function ‘int main()’:
main.cpp:8:23: error: conversion from ‘std::vector*’ to non-scalar type ‘std::vector’ requested
8 | vector<int> vec = new vector<int>;
| ~~~~~~~~~~~~~~~ |
A type is scalar if has no user visible components and non-scalar otherwise.
vector<int> *v = new vector<int>;
delete v;Eg 2: (new/delete)
int *ptr = new int;
delete ptr;Eg 2: (new[]/delete [])
int *ptr = new int[10];
delete []ptr;Note: delete always expects the ptr to be int * here but not int.
Problem 1
#include <iostream> using namespace std; int main() { int ptr = new int; delete ptr; return 0; }
In function 'int main()':
Line 7: error: invalid conversion from 'int*' to 'int'
compilation terminated due to -Wfatal-errors. |
Sample Program
Click here to debug.
#include <iostream> #include <vector> using namespace std; int functionFoo (int var) { int *vec = new int; cout << "Variable is " << var << endl; // Assign value to a raw pointer from a variale *vec = var; // Assign value to a variable from a raw pointer int res = *vec; cout << "Vec is " << *vec << " res " << res << endl; return res; } int main() { int ret = functionFoo(23); cout << "return " << ret << endl; return 0; }
You can read more about memory layout here.
Calloc
Realloc
Free
Deallocation of memory by the Operating System (OS) is a way to free the Random Access Memory (RAM) of finished processes and allocate new ones. We all know that the computer memory comes with a specific size. A process has to be loaded into the RAM for its execution and remains in the RAM until its completion. Finished processes are deallocated or removed from the memory and new processes are allocated again. This is how the OS works with allocation and deallocation.
While freeing, it just looks the extra information to find out how big the block is. This extra information is placed just before the address, given in free. This extra information will hold the size of the memory allocated including requested memory and header size.
Example:
My requirement is to copy the IPv6 address from the array of chars to the structure's member which is another string. I've kept on adding lines to solve the problem by verifying copying IPv6 address to another array using strcpy, memcpy, and etc.,
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
struct ipv6hdr /* total size 269 bytes */
{
unsigned char flow_lbl[3]; /* 3 bytes */
unsigned int payload_len; /* 8 bytes */
unsigned char nexthdr; /* 1 byte */
unsigned char hop_limit; /* 1 byte */
char saddr[128]; /* 128 bytes */
char daddr[128]; /* 128 bytes */
};
int main()
{
char dst_addr[135] = "bc00:2::1";
char addr_store[135];
char *buf = NULL; /* Memory is not allocated */
struct ipv6hdr *ip6_hdr = (struct ipv6hdr *)buf;
struct ipv6hdr *ip6_hdr1 = NULL;
struct ipv6hdr *ip6_hdr2 = NULL;
/* Copying a string to another string using strncpy*/
strncpy(addr_store, dst_addr, strlen(dst_addr));
printf ("Using strncpy: Dst Addr %s Copied Addr %s\n", dst_addr, addr_store);
/* Copying a string to another string using memcpy*/
memset(addr_store, 0, sizeof(addr_store));
printf ("After memset: Dst Addr %s Copied Addr %s\n",
dst_addr, addr_store);
memcpy(addr_store, dst_addr, strlen(dst_addr));
printf ("Using memcpy: Dst Addr %s Copied Addr %s\n",
dst_addr, addr_store);
fflush(stdout);
/* Copying a string to a structure member */
//memcpy(ip6_hdr->daddr, dst_addr, strlen(dst_addr));
//printf ("IPv6 Hdr Addr %s Dst Addr %s\n", ip6_hdr->daddr, dst_addr);
/* Memory is not allocated to structure ipv6hdr */
char trrt6_buf[12]; /* Memory is allocated for this array */
ip6_hdr = (struct ipv6hdr *)trrt6_buf;
memcpy(ip6_hdr->daddr, dst_addr, strlen(dst_addr));
printf ("Memory Allocated: IPv6 Hdr Addr %s Dst Addr %s\n",
ip6_hdr->daddr, dst_addr);
/* 1000 bytes Memory allocation using malloc */
ip6_hdr1 = (struct ipv6hdr *)malloc(sizeof(char) * 1000);
memset(ip6_hdr1, 0, sizeof(struct ipv6hdr *));
memcpy(ip6_hdr1->daddr, dst_addr, strlen(dst_addr));
printf ("Malloc: IPv6 Hdr Addr %s Dst Addr %s\n",
ip6_hdr1->daddr, dst_addr);
/* Memory allocation using malloc */
ip6_hdr2 = (struct ipv6hdr *)malloc(sizeof(struct ipv6hdr));
memset(ip6_hdr2, 0, sizeof(struct ipv6hdr *));
strncpy(ip6_hdr2->daddr, dst_addr, strlen(dst_addr));
printf ("Malloc: IPv6 Hdr Addr %s Dst Addr %s Size %d\n",
ip6_hdr2->daddr, dst_addr, sizeof(struct ipv6hdr));
return 0;
}Output:
|
I'm not sure why the size of the structure is 268 and not 269. Please let me know in comments if you are aware.
In line 'ip6_hdr2 = (struct ipv6hdr *)malloc(sizeof(struct ipv6hdr));', the malloc returns 'void *', but ip6_hdr2 holds only 'struct ipv6hdr *'. So, typecasting (struct ipv6hdr *) is needed.
List of Questions which can be asked in Interview
What is Memory Leak?
If we used a bunch of memory for use and never give it back to OS, then it will leak out. If this goes long enough, the computer may crash/sieze.
We can avoid memory leaks using the following methods:
1. The best way to avoid memory leaks in C++ is to have as few new/delete calls at the program level as possible – ideally NONE. Anything that requires dynamic memory should be buried inside an RAII object that releases the memory when it goes out of scope. Resource Acquisition Is Initialization (RAII) allocate memory in constructor and release it in destructor, so that memory is guaranteed to be deallocated when the variable leave the current scope. Eg: shared_ptr, unique_ptr.
RAII ensures that object live independently of its scope.
2. Instead of managing memory manually, try to use smart pointers where applicable. It avoids the overhead of manual memory deallocation.
3. Allocate memory by new keyword and deallocate memory by delete keyword and write all code between them.
Example 1:
int* sptr = new int; int* tptr = sptr; cout << *tptr; delete sptr;
This program has no memory errors, because when the sptr's memory is released, tptr is also will be released.
Even if the last line is replaced with tptr, the memory will be released as both sptr and tptr points to same location.
It's good practice to set the sptr and tptr to nullptr after delete.
Example 2:
int* ptr; ptr = new int[10]; // 10*4 bytes allocated ptr[9] = 1; ptr = new int[20]; // 20*4 bytes allocated and ptr points to another location ptr[19] = 2; delete [] ptr; // The last location where ptr points is deallocated cout << ptr[9];
This program has memory leak because the first 10*4 bytes not released.
It tries to access the memory which is no longer owned at the last line.
Why we need to allocate memory?
1. If we want to store data, we need to give the pointer memory.
2. Pointer pointing to a NULL memory still indicates memory is not allocated.
What happens if we don't explicitly allocate memory?
Segmentation fault. (Uncomment the commented code and check.)
What is memory/heap fragmentation?
If you make a lot of calls to malloc() and free() with varying memory size requests, it could, in theory, cause a condition called "memory fragmentation", where there is enough space in the free-store to allocate your requested memory block, but not enough contiguous space for the size of the block you've requested. Thus the call to malloc() fails, and you're effectively "out-of-memory" even though there may be plenty of memory available as a total amount of bytes in the free-store.
How to avoid heap fragmentation?
Allocator such as dlmalloc handles fragmentation better than Win32 heaps.
What will you do, if you want to increase the size of the array?
Before compilation, I could definitely change. But, since this array demands more memory, it might need more memory in future as well. So, instead of going for fixed size array, we can allocate memory dynamically. So, if the array needs more memory, we could reallocate it. Because dynamic memory has the luxury of increasing and decreasing the memory at run time.
How does free know how much to free ?
While freeing, free() will get to know the starting address of the allocated memory for the passed pointer. It simply checks the address before it which usually holds the size of the memory allocated including requested memory and header size.
What is dangling pointer?
A dangling pointer is a non-NULL pointer which points to unallocated(already freed) memory area.
A dangling pointer is a pointer that points to invalid data or to data which is not valid anymore, for example:
Class *object = new Class();
Class *object2 = object;
delete object;
object = nullptr;
// now object2 points to something which is not valid anymoreWhat is memory/heap corruption?
- Overwrite the memory will cause the memory corruption, or in other words, writing outside the bounds of an array.
- Double-freeing dynamically allocated memory
- Writing to an uninitialized pointer.
Memory when altered without an explicit assignment due to the inadvertent and unexpected altering of data held in memory or the altering of a pointer to a specific place in memory.
Buffer overflow:
Example 1:
Overwrite beyond allocated length - overflow.
1 | char *a = malloc(128*sizeof(char)); |
2 | memcpy(a, data, dataLen); // Error if dataLen too long. |
Example 2:
Index of array out of bounds: (array index overflow - index too large/underflow - negative index)
1 | ptr = (char *) malloc(strlen(string_A)); // Should be (string_A + 1) to account for null termination. |
2 | strcpy(ptr, string_A); // Copies memory from string_A which is one byte longer than its destination ptr. |
What is double free?
Freeing twice the memory will lead to double free.
Freeing memory which has already been freed. Also applies to delete.
Freeing a pointer twice:
1 | char *a = malloc(128*sizeof(char)); |
2 | free(a); |
3 | ... Do stuff |
4 | free(a); |
We can avoid double fee using the following method:
Its a common mistake to free() or delete allocated memory more than one. It may help to insert something like *var = NULL after such calls, and to check for != NULL when calling free. Although in C++ its legal to call delete with a NULL variable, calling C - free() will fail.
- Also a common problem is to confuse delete and delete [].
- Variables allocated with new must be released with delete.
- Array allocated with new [] must be released with delete[].
Incorrect use of delete: The delete must match the use of new.
The pairing is new/delete and new [] / delete[]
1 | ClassABC *abc_ptr = new ClassABC[100]; |
2 | ... |
3 | delete [] abc_ptr; |
Also make sure not to mix C- style memory management (malloc, calloc, free) with C++ style memory management (new/delete). In legacy code often both are mixed, but things allocated with the one can not be freed with the other.
How malloc knows what is the next free memory?
References
https://www.youtube.com/watch?v=HPDBOhiKaD8
https://stackoverflow.com/questions/60871/how-to-solve-memory-fragmentation
https://study.com/academy/lesson/how-to-allocate-deallocate-memory-in-c-programming.html
https://stackoverflow.com/questions/1518711/how-does-free-know-how-much-to-free
https://stackoverflow.com/questions/48757030/why-use-a-new-call-with-a-c-vector
Comments
Post a Comment