Memory allocations are slow. If memory isn’t managed carefully, gc will cause application freezes. While GC is relevant for languages such as java where memory is managed automatically by the virtual machine, the same concept also applies to languages like C & C++ where memory is managed in code.
To avoid performance penalties from memory allocation and the resulting affects of GC it is crucial that we design objects via memory pressure in mind.
Some lessons on memory optimizations:
- Design objects to have references to common immutable data that can be shared across the objects thereby reducing need for deep copies.
- Its common to leverage containers such as hash maps in java. However, when allocating large maps they take up a lot of extra space due to need for managing internal space ‘MapEntry’. Its more efficient to design objects based on arrays and write code as data transformations on arrays. This also helps in execution as data can be fetched via array offsets rather than need to compute a hash and then searching for the data.
- Utilize object pools. In cases where the type of data that needs to be used per event is similar, we can store the immutable objects in a guava-based object pool backed by soft references. This helps avoid repeated memory allocations for common objects and thus reduces pressure on GC. I have found this approach to be very useful in our architecture while trying to run our applications on machines with lower memory.
- Understand and better utilize tlab (thread local access buffer). I have found TLAB allocations to be 10 times or more faster than allocations outside of tlab. This is because allocations outside of tlab need to be synchronized across threads. Don’t be alarmed if you have some percentage of allocations outside of TLAB, this is normal. When working with huge objects, I typically design for them to be processed asynchronously outside of the critical path.
- Efficiently manage data transformations. Code is typically working on data types and transforming the data across types as it executes. Many times the data transformations are repetitive, I,e transforming from epoc long date to a date time object on which you can apply some date based calculations. In these cases, it is best to design an object transformation utility that can via using an lru mapping cache return the transformed object efficiently.
Here is a story from real world experience. Many years ago, I designed and wrote a classification service that worked on an array-based object. The array-based data object was built similar to a database table on which filters and calculations could be applied similar to sql queries in memory using a classification engine that was constructed in the application. When the application started it had about 20 different attributes that it worked on and keeping them in a single array made sense. However, as years went by the application grew to have over 500 attributes which were still stored in the same array-based object. The application started to consume too much memory and major parts of it had to be modified to decrease the memory pressure. A better decision on my part would have been to design the main classification data object as a series of data objects that would be linked together via references rather than a single array-based object that held references to respective data at array indexes.
See below table showing how the object was originally designed. I have only pointed 6 columns below but the actual number of items in the array were a lot more which continued to grow over years. The business data in reality existed such that many of the values in the array were just null (not applicable) but the fact that a null reference existed in the array it kept using space for the reference.
Data object
| Object name | Object type | Company Location | Company Rating | Quantity | Direction |
| Apple | Stock | USA | A++ | 100 | Buy |
A better design would have been to design the object as an array of data objects each of which had its data in an array, for example see below.
Data object
| Company Info | Trade information |
| Reference to the object which represents the item being traded | Reference to the object which references the trade |
Company Info
| Item name | Item type | Item Location | Item Rating |
| Apple | Stock | USA | A++ |
Trade Info
| Quantity | Direction |
| 100 | Buy |