Hello SYB DBAs,
you may be noticed a message in your DB log which looks like this:
"Could not allocate memory using Huge Pages. Allocated using regular pages. For better performance, reboot the server after configuring enough Huge Pages."
Don't worry about it, if no Huge Pages are available, the system automatically switch back to the normal page size of the OS. For example in Linux the regular memory page size is 4KB.
The huge page size you can find out via command:
cat /proc/meminfo | grep Hugepagesize Hugepagesize: 2048 kB
Hugepages speed up the memory management.
Hugepages allocated in one piece and can't be paged out into the paging space.
So you can ensure that the DB memory never be paged out.
Please read also this notes about HugePages and Sybase/SAP ASE:
2021541 - SAP ASE runs slow due to table scans with regular page allocation
1805750 - SYB: Usage of huge pages on Linux Systems with Sybase ASE
For example if the sybase instance need 10GB without Hugepages:
10*1024*1024/4kb=2621440 (4kb Pages) Every Page needs a 8byte entry in the page table 2621440 * 8byte = 20971520 => 20MB
For example if the sybase instance need 10GB with Hugepages:
10*1024*1024/2048kb=5120 (2mb Pages) Every Page needs a 8byte entry in the page table 5120 * 8byte = 40960 => 40KB
This values need to be multiplicated with the user processes, but the shared memory must be considered!
If you need more information about the Page Table, TLB and Huge Pages you should have a look at Stefan Koehlers blog which is regarding oracle but however the facts are the same:
If you want to use HugePages follow this steps as root
1) calculate the needed memory
select cast(sc.name as varchar(30)) as name, (scc.value*2/1024) as MB from sysconfigures sc, syscurconfigs scc where sc.name like 'total%memory' and sc.config = scc.config go name MB ------------------------------ ----------- total logical memory 6028 total physical memory 6400
=> Round the ASE "max memory" (total physical memory) setting up to the next closest 256MB, divide by the Hugepagesize, and configure at least that many huge pages. "
=> 6400 / 256mb = 25 => OK
2) calculate the needed HugePages
When ASE uses huge pages, it will allocate memory to the nearest multiple of 256Mb so don’t configure the ASE exactly to the maximum number of huge pages, but leave a small amount unused (e.g. 300/400 Mb). In case a small increase in size is needed due to the change of a Sybase config you will not get into trouble.
6400+400=6800 6800*1024/2048=3400 vi /etc/sysctl.conf vm.nr_hugepages=3400
3) activate the kernel settings, if enough memory is available
sysctl -p
if the memory is not free, restart the server or add memory
4) Allow Sybase ASE owner to make use of available HugePages
vi /etc/security/limits.conf<Sybase ASE OS owner> soft memlock unlimited<Sybase ASE OS owner> hard memlock unlimited
5) check the config
cat /proc/meminfo | grep Huge HugePages_Total: 3400 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB
6) start the Sybase/SAP ASE and you will see this message in the logs
"Allocated memory using Huge pages."
7) after DB start
cat /proc/meminfo | grep Huge HugePages_Total: 3250 HugePages_Free: 1655 HugePages_Rsvd: 1605 HugePages_Surp: 0 Hugepagesize: 2048 kB
If you get the same error as before extend the HugePages.
more offical info:
1805750 - SYB: Usage of huge pages on Linux Systems with Sybase ASE
Before and after process details
Here are the before and after effects regarding the dataserver (sybase DB process under unix) after a fresh restart:
Before:
pmap <sid> START SIZE RSS PSS DIRTY SWAP PERM MAPPING 0000000000400000 32736K 15460K 15460K 0K 0K r-xp /sybase/SMP/ASE-16_0/bin/dataserver 00000000025f7000 5692K 2664K 2664K 216K 0K rwxp /sybase/SMP/ASE-16_0/bin/dataserver 0000000002b86000 2488K 1196K 1196K 1196K 0K rwxp [heap] 0000000142df4000 6291456K 1507196K 1507196K 1507172K 0K rwxs /SYSVba156435 00007fffed33e000 20K 16K 2K 0K 0K r-xp /lib64/libnss_dns-2.11.3.so [...] Total: 6507456K 1530568K 1528772K 1509564K 0K
ps auxw | head -1; ps auxw | grep <sid> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND sybsmp 31464 15.5 9.2 6507456 1516852 ? Ssl 08:56 0:24 /sybase/SMP/ASE-16_0/bin/dataserver -sSMP -d/sybase/SMP/sybsystem/master.dat -e/sybase/SMP/ASE-16_0/install/SMP.log -c/sybase/SMP/ASE-16_0/SMP.cfg -M/sybase/SMP/ASE-16_0 -i/sybase/SMP -N/sybase/SMP/ASE-16_0/sysam/SMP.properties
=> RSS = 1,5GB
=> VSZ = 6,5GB
After:
pmap <sid> START SIZE RSS PSS DIRTY SWAP PERM MAPPING 0000000000400000 32736K 16096K 16096K 0K 0K r-xp /sybase/SMP/ASE-16_0/bin/dataserver 00000000025f7000 5692K 2664K 2664K 216K 0K rwxp /sybase/SMP/ASE-16_0/bin/dataserver 0000000002b86000 2648K 1244K 1244K 1244K 0K rwxp [heap] 00002aaaaac00000 6291456K 0K 0K 0K 0K rwxs /SYSVba156435 00007fffed33e000 20K 16K 0K 0K 0K r-xp /lib64/libnss_dns-2.11.3.so 00007fffed343000 2044K 0K 0K 0K 0K ---p /lib64/libnss_dns-2.11.3.so 00007fffed542000 4K 4K 4K 4K 0K r-xp /lib64/libnss_dns-2.11.3.so 00007fffed543000 4K 4K 4K 4K 0K rwxp /lib64/libnss_dns-2.11.3.so 00007fffed544000 48K 20K 0K 0K 0K r-xp /lib64/libnss_files-2.11.3.so 00007fffed550000 2044K 0K 0K 0K 0K ---p /lib64/libnss_files-2.11.3.so [...] Total: 6507616K 24056K 22094K 2440K 0K
vmsaplnx02:~ # ps auxw | head -1 ; ps auxw | grep <sid> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND sybsmp 1716 9.6 0.1 281536 22712 ? Ssl 09:12 0:20 /sybase/SMP/ASE-16_0/bin/dataserver -sSMP -d/sybase/SMP/sybsystem/master.dat -e/sybase/SMP/ASE-16_0/install/SMP.log -c/sybase/SMP/ASE-16_0/SMP.cfg -M/sybase/SMP/ASE-16_0 -i/sybase/SMP -N/sybase/SMP/ASE-16_0/sysam/SMP.properties
=> RSS = 23MB
=> VSZ = 282MB
If you compare the RSS/VSZ of the 2 ouput you will see the benefit.
Hope this howto helps you to use your memory ressources optimal.
Thanks for reading and sharing!
Please feel free to ask or get in contact directly if you need if you need assistance.
Best Regards,
Jens Gleichmann
Technology Consultant at Q-Partners Consulting und Management GmbH (www.qpcm.eu)
References
Virtual memory - Wikipedia, the free encyclopedia
Resident set size - Wikipedia, the free encyclopedia
2021541 - SAP ASE runs slow due to table scans with regular page allocation
1805750 - SYB: Usage of huge pages on Linux Systems with Sybase ASE