503 Error¶

503 errors are often caused by a malfunction in PHP. It is rare for the web server to be the cause of a 503 error, but just to be safe, you should rule that out first thing by checking whether the error occurs with Apache as well.

Does it Occur with Apache?¶

If your server is running on an Apache control panel, such as cPanel or Plesk, there is an easy way to determine whether an issue is caused by your LiteSpeed server:

1. Temporarily switch to Apache
2. Repeat the steps that originally led to the issue.

Are you able to reproduce the error under Apache?

NO¶

If the error could not be triggered under Apache, then the problem is likely to be a LiteSpeed server issue. Please open a ticket from your client area or email support@litespeedtech.com and provide as many details as possible, so that we may assist you.

YES¶

If Apache experiences the same problems, then the issue is not a LiteSpeed Web Server issue.

Switch back to LiteSpeed Web Server, and keep reading! Even though it is not a server problem, we have provided the following troubleshooting documentation that we hope will help you to find the solution.

$ sudo touch /usr/local/lsws/admin/tmp/.lsphp_restart.txt
$ sudo systemctl restart lsws

Quick Troubleshooting Guide¶

Before engaging us for Premium Support service, you may like to try a few of these simple steps to see if you can fix the error yourself:

• Check the phpinfo page of the problematic user account. (Create a phpinfo.php file if no phpinfo page exists.)
• Generally, the server's error_log or stderr.log should provide some hints as to the problem. For example, it may be a memory issue.
• Disable opcode cache for the PHP version in use and verify from the phpinfo page.
• Disable unsafe PHP extensions such as ZendGuardLoader, Suhosin, ionCube, etc.
• Check disk space.
• Try increasing the PHP memory limit (you can verify from phpinfo.php) and lsphp external app memory limit.
• If you're using CloudLinux:
  • Check the LVE memory limit to see if it needs to be increased.
  • Check the LVE process limit to see if it needs to be increased.
  • Try to force an update of CageFS.

These are just a few simple quick fixes for you to try first. If they don't help, you can refer to the following log files for detailed steps/reasons, or engage our support as mentioned earlier.

Check Log Files¶

There are many different log files that may help you determine the cause of the 503 error. In this section we will discuss the different log files, their locations, and what information they provide.

When troubleshooting 503 errors it is ideal to check these logs in the following order:

1. Standard Error Log
2. Web Server Error Log
3. System Log
4. PHP Error Log

Standard Error Log stderr.log¶

The stderr.log file contains standard errors generated while PHP is running. It can usually be found in either the /var/log/apache2/ or/usr/local/lsws/logs/ directory. This log is usually the most helpful when determining why PHP is throwing 503s.

[STDERR] fork() failed, please increase process limit: Cannot allocate memory

Web Server Error Log¶

This error log is generated by the web server. It can usually be found in either the /var/log/apache2/ or /usr/local/lsws/logs/ directory. The web server error log may provide helpful hints as to whether the web server caused PHP to fail.

[INFO] [319934] [1.1.1.1:49873-1#APVH:lsapi] connection to [uds://tmp/lshttpd/APVH-php56.sock] on request #0, confirmed, 0, associated process: 0, running: 0, error: Too many open files!

The following very important command should give you ideas on how many 503 errors have been generated and wheather there are any new 503 errors since your last attempted fix by checking the timestamp. However, most of the time, the error log doesn't give you the reason why the 503 error happened. It only shows you when it happened and with which domain it happened.

grep oops /etc/apache2/logs/error_log

System Log¶

This error log is generated by the operating system. It can usually be found as either /var/log/messages or /var/log/syslog. This log is helpful if PHP is being affected by the operating system.

lfd[18304]: *User Processing* PID:18264 Kill:1 User:xxxxx VM:538(MB) EXE:/usr/local/lsws/fcgi-bin/lsphp-5.4.42 CMD:lsphp5

PHP Error Log¶

This error log is defined in php.ini and you can check phpinfo.php for the php error_log defined. By default, cpanel will set it to error_log. It is generated by PHP when it detects an error, warning, etc., depending on your log level in PHP. It is usually generated inside of the same folder that the PHP script runs in.

Fatal error: Call to undefined function my_function() in /home/mysite/public_html/test.php on line 2

These types of errors are usually found inside of the PHP application you are trying to run. It is best to forward these logs to the developer of the PHP application.

Troubleshooting¶

Basic¶

Display Errors¶

In a production environment it is common to disable PHP application errors from displaying in order to secure the server in the event of a issue. This however is counter productive when trying to diagnose 503 errors.

If you are experiencing 503 errors for a specific site/domain, try adding the following right after the opening <?php tags, to the same script with the 503 error:

<?php # The opening PHP tag
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

When you visit the web page now, it should show an error like this:

Fatal error: Call to undefined function my_function() in /home/mysite/public_html/test.php on line 2

Normally this has nothing to do with the web server and is actually an issue with the code. We suggest checking Stack Overflow or contacting the developer of the software for a fix.

PHP Info Page¶

The PHP info page gives us a lot of detail about your PHP installation. This page is a very basic file that should not throw a 503 error.

<?php
echo phpinfo();
?>

Here is a list of some of the useful information the PHP info page gives us:

• Extensions
• Memory Limit
• Execution Time
• Other Settings

With this information, you can see if OPCache is enabled, what the memory limit is, which extensions are loaded, etc; all useful information when looking for the cause of a 503 error.

This page can also be used to compare differences between Apache's and LiteSpeed's PHP implementations.

Extensions¶

These are a handful of PHP extensions that have been known to cause issues, especially when they are outdated:

• Imunify360
• ZendGuardLoader
• Suhosin
• ionCube

We recommended you regularly check for updates for these extensions. If you are already using the latest versions, try to disable them and see if the problem persists.

If you disable these plugins and the 503 error is still being shown, disable all other extensions that were not enabled by default, and check again.

OPCache¶

OPCache is the process of storing bytecode in shared memory so PHP does not need to process the same calls over and over, but it can occasionally cause issues. When troubleshooting 503 errors, it is best to have OPCache disabled.

Common OPCache Extensions:

• OPCache
• APC/u
• XCache
• eAccelerator

To disable the OPCache extensions follow the extensions guide.

[STDERR] zend_mm_heap corrupted

Disk Space¶

PHP will sometimes store files on your server in order to function properly. If your disks become full, especially the partition that houses /tmp/, it could be the cause of your 503 error.

To check your disk space usage you can run $ sudo df -h in a terminal.

$ sudo df -h
Filesystem                          Size  Used Avail Use% Mounted on
devtmpfs                            901M     0  901M   0% /dev
tmpfs                               915M   96K  915M   1% /dev/shm
tmpfs                               915M   89M  827M  10% /run
tmpfs                               915M     0  915M   0% /sys/fs/cgroup
/dev/mapper/cl_centos                29G  4.2G   25G  15% /
/dev/vda1                           976M  184M  726M  21% /boot
tmpfs                               183M     0  183M   0% /run/user/0

If you have full disks, clean up unused and old files. This should solve the 503 error.

Memory Limit¶

When PHP reaches its allocated memory limit, it will crash. There are 2 different memory limits that PHP must adhere to: its internal memory limit, set by php.ini, and LiteSpeed's memory limit, set inside of the web server for when the PHP process spawns.

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 12864 bytes)

[STDERR] fork() failed, please increase process limit: Cannot allocate memory

PHP Memory¶

If you get a memory error like the one above, the first step in troubleshooting it is to increase the memory limit. Adjust the following setting inside of php.ini:

memory_limit = 100M

To figure out where your php.ini file is located, and find out your current memory limit, you can create a PHP info page.

<?php
echo phpinfo();
?>

Sometimes scripts may append lines to .htaccess to override the PHP settings. This can override what is set in your php.ini file.

Here is an example of a memory_limit entry inside of .htaccess:

php_value memory_limit xxxM

You can add/delete/modify this line inside of your .htaccess file to suit your needs.

CloudLinux Memory Limit¶

If you are running CloudLinux and Lightweight Virtual Environment (LVE) it is possible that you are hitting the limits set by that account.

Here shows accounts that have hit a limit, depending on what limit the account has hit you should adjust the limit. To adjust the limits follow CloudLinux's guide.

litespeed PHP Process Memory¶

If you have increased the memory limit in php.ini and confirmed you are not hitting a LVE limit then it is possible LSPHP needs to increase its virtual memory.

1. Go to your Web Admin console ( https://YOUR_SERVER_IP:7080 ) and log in.
2. Next, navigate to Configuration > Server > PHP.
3. Now click Edit for PHP Handler Defaults.
4. Change the following settings, then click Save.
   • Memory Soft Limit: 4097M
   • Memory Hard Limit: 4098M
5. Now that everything has been set up, navigate to Actions and press the icons next to Restart Detached PHP Processes. Then Apply Changes / Graceful Restart.

Max Execution Time¶

If you have a max execution time that is not high enough to handle long running requests then you may get the following error in your web server error log: error: Connection reset by peer! and have a value of 0, meaning that the process never finished the request. These errors will often be explained in your stderr.log. If the stderr.log does not provide any insight then you can try to increase the max execution time by following the steps in the Control guide.

PHP suEXEC Max Conn¶

In stderr.log, you may see something like the following:

[30117] Reached max children process limit: 10, extra: 3, current: 13, busy: 13, please increase LSAPI_CHILDREN.

CSF/LFD¶

If you have CSF/LFD installed they may be causing the 503 error. These programs may be killing the process before it finishes. CSF/LFD has two settings that can kill a LSPHP process: PT_USERTIME and PT_FORKBOMB.

Kill¶

If a LSPHP process takes too long to complete it is possible that the PT_USERTIME feature will be called to kill the process. If it does kill the process you may receive an email or an entry in /var/log/lfd.log with the following message:

lfd[18304]: *User Processing* PID:18264 Kill:1 User:xxxxx VM:538(MB) EXE:/usr/local/lsws/fcgi-bin/lsphp-5.4.42 CMD:lsphp5

This means that we need to whitelist the LSPHP process from CSF/LFD to prvent this from happening.

To do this, run the following commands in a terminal:

$ sudo echo "pexe:/usr/local/lsws/fcgi-bin/lsphp.*" >> /etc/csf/csf.pignore
$ sudo csf -r; sudo service lfd restart

Forkbomb¶

In rare cases, LSPHP will be killed by PT_FORKBOMB for having too many processes spawned at any given time. This will, in return, end up sending the signal 9/SIGKILL to the main Litespeed process. To stop csf/lfd from killing the LSPHP and LiteSpeed processes, the following settings inside of /etc/csf/csf.conf should be adjusted:

PT_LIMIT - Default: 10
PT_USERPROC - Default: 10
PT_FORKBOMB - Default: 0
PT_INTERVAL - Default 60

These settings should be adjusted on a per case basis as each scenario is different. They should be increased slightly higher than the current values until the issue stops occurring. For the changes to take effect, both csf and lfd need to be restarted. This can either be done via the WHM plugin or through a shell terminal by using the following command:

$ sudo csf -r; sudo service lfd restart

Extension Mismatch¶

Running LSPHP extensions that are not built for the version of LSPHP you are using may cause a 503 error. If this is the case, you will see the following messagin inside of a stderr.log:

Warning: PHP Startup: imap: Unable to initialize module
Module compiled with module API=20090626
PHP    compiled with module API=20100525
These options need to match in Unknown on line 0

To fix this, you must rebuild the module and/or PHP, making sure to use a version of PHP that works with the module. Also, make sure that the correct extension path is used.

Extension Loading Order¶

With PHP, some extensions require other extensions be loaded first in order to work efficently. This is accomplished by changing the extensions's .ini file from mysql.so to 20-mysql.so. The 20- value is the loading priority. It can be changed from 01- to 99-, with lower values loading before the higher values. You can find more information on the PHP site.

If this is the cause of the issue, then you should see the following entry in your stderr.log.

PHP Warning:  PHP Startup: Unable to load dynamic library

Open Files Limit¶

If you see the following entry inside of your web server error long, then your system is maxed out of the amount of files it can open.

[INFO] [319934] [1.1.1.1:49873-1#APVH:lsapi] connection to [uds://tmp/lshttpd/APVH-php56.sock] on request #0, confirmed, 0, associated process: 0, running: 0, error: Too many open files!

To check your current limit, run the following command:

$ sudo ulimit -n

It will show you your system's current limit. To fix this issue, increase the limit by running:

$ sudo ulimit -n <Increased_Number>

Now you should no longer see a 503 error because LSPHP can open more files on your system.

Resource temporarily unavailable¶

If you are using WordPress, you may see this error generated when working with images or graphics:

libgomp: Thread creation failed: Resource temporarily unavailable

PHP code issue¶

Bad PHP code or a buggy script may create too many PHP connections or a loop event, which can cause 503 errors. You can strace the PHP process to find out more, and then optimize your PHP code or script.

Advanced¶

The comprehensive guide above lists the most likely reasons you'd receive a 503 error, but in some circumstances, it may be difficult to locate the root cause. If PHP crashes, you can enable a core dump and use GDB to debug the core file. Even if you find an internal PHP bug causing the crash, there are limited actions you can take. You can disable the module or change to another PHP version without the bug. Reporting the bug to PHP is an option, but a solution may not be provided in a timely manner.

If there is a core file generated and you've tried all of the above methods without finding a solution, try to strace the PHP process for a hint. Just be prepared that the strace log may be extremely long and overwhelming. You likely need some knowledge and experience to understand what the log entries mean; it can even be a challenge locating the relevant sections of the log.

Strace¶

Use the following command to strace the PHP process to find out what happened:

strace -tt -T -f -p <pid>

This command will show you what PHP is doing and where the issues is.

[INFO] [1.1.1.1:48506] connection to [/tmp/lshttpd/APVH_vhost_Suphp.sock] on request #0, confirmed, 1, associated process: 791228, running: 1, error: Connection reset by peer!

strace -tt -T -f -p 791228

while true; do if mypid=`ps aux | grep $USERNAME | grep lsphp | grep $SCRIPTNAME | grep -v grep | awk '{print $2; }' | tail -1`; then strace -tt -T -f -p $mypid; fi ; done

Core Files¶

If you find that PHP has crashed (as demonstrated by the process being killed by signal other than value 15), a core dump will allow you to look further into the cause of the crash. These core files are usually generated inside of the folder that PHP crashed with.

Enable Core Files¶

This section assumes you have core dumps enabled for centOS or Ubuntu already.

1. Go to your Web Admin console ( https://YOUR_SERVER_IP:7080 ) and log in.
2. Next, navigate to Configuration > Server > PHP.
3. Now click Edit for PHP Handler Defaults.
4. Change the following settings then click Save.
   • Environment: LSAPI_ALLOW_CORE_DUMP=1
5. Now that everything has been set up, navigate to Actions and press the icons next to Restart Detached PHP Processes. Then Apply Changes / Graceful Restart.

Analayze Core Files¶

GNU Debugger (GDB) uses the syntax gdb <path/to/lsphp/binary> <path/to/core/file>. The <path/to/lsphp/binary> is the path to the PHP binary that the PHP application was using. The path to the core file is the location of the core file.

After running the above command, you will see a prompt with (gdb) then you will want to type bt to view the backtrace then hit enter.

This will generate output like the following:

gdb <path/to/lsphp/binary> <path/to/core/file>
Program received signal SIGSEGV, Segmentation fault.
0x000000000061c91b in ?? ()
(gdb) bt
#0 0x000000000061c91b in ?? ()
#1 0x0000000000641ac3 in zend_stack_push ()
#2 0x000000000060bbf9 in ?? ()
#3 0x0000000000611515 in lex_scan ()
#4 0x000000000061fb60 in ?? ()
#5 0x0000000000608223 in ?? ()
#6 0x0000000000614965 in compile_file ()
#7 0x00007fd3c99eda21 in ?? () from /opt/alt/php55/usr/lib64/php/modules/phar.so
#8 0x00007fd3d0993359 in ?? () from /opt/alt/php55/usr/lib64/php/modules/opcache.so
#9 0x00007fd3d0994187 in ?? () from /opt/alt/php55/usr/lib64/php/modules/opcache.so
#10 0x00000000006b54da in ?? ()
#11 0x00000000006b68e8 in execute_ex ()
#12 0x000000000064239c in zend_execute_scripts ()
#13 0x00000000005e2fb0 in php_execute_script ()
#14 0x00000000006f29ff in ?? ()
#15 0x00000000006f2c5c in ?? ()
#16 0x00000000006f2f85 in ?? ()
#17 0x00007fd3d37cdd1d in __libc_start_main () from /lib64/libc.so.6
#18 0x0000000000424bc9 in _start ()

Here we can see that the some of the modules that loaded when it crashed were phar.so and opcache.so. To try to troubleshoot this, you would want to disable phar.so and opcache.so to figure out which module is causing the issue.

(gdb) frame 13
(gdb) print (char *)(executor_globals.function_state_ptr->function)->common.function_name
(gdb) print (char *)executor_globals.active_op_array->function_name
(gdb) print (char *)executor_globals.active_op_array->filename

Auto Fix 503¶

In LiteSpeed Web Server's WebAdmin Console, there is a feature to Auto Fix 503 Error. It specifies whether to try to fix the 503 Service Unavailable error by restarting the server gracefully. A 503 error is usually caused by malfunctioning external applications and a web server restart can often fix the error temporarily. If Auto Fix 503 Error is set to Yes, the server will restart automatically whenever there are more than thirty 503 errors within a 30 second span. This feature is disabled by default, but you can enable it by navigating to LSWS Admin > Server > General and setting Auto Fix 503 Error to Yes.

Auto Fix 503 Error is not a permanent fix. You should not rely on this feature to fix recurring 503 errors. Going through the above steps to find a permanent fix is always recommended.