Getting Your CGI Scripts Working

This tutorial has been developed to help you get your CGI script working. It focuses on common problems associated with setting up CGI scripts. There are some links to other helpful resources at the end. You should check these out if you cannot find the answers you need here.

What follows is a sort of checklist to help you solve the problem quickly yourself. It is divided into four parts:

Before Uploading to the Server

1. Does your script compile?

Like any Perl program, a CGI script needs to compile successfully before it will run. Until it does this you can forget everything else. To test this you need a command prompt and a working Perl interpreter. For information on how to build a Perl interpreter or to get a pre-built Perl for your platform, Installing and Adding to Perl on PerlMonks.

Once you have a perl you can access from a command prompt, type:

perl -c myscript.pl

What this does is compile (but not run) your script. You should see this:

perl -c myscript.pl
myscript.pl syntax OK

Here are some common errors and solutions:

No Perl!

perl -c test.pl
Bad command or file name

This is telling you that your operating system can not find the Perl executable. Either it is not installed and/or it is not included in your path environment variable. So install perl and either include it in your path or type the full path to the executable. How you find perl depends on your system. On UNIX-like systems, type 'which perl' at the command prompt to get the path (usually /usr/bin/perl). On Windows, do a file search (Start > Find) for perl.exe to get the path (but it's usually in C:\Perl\bin\).

A bit of background: the PATH environment variable is a list of directories separated by colons (UNIX shells) or semicolons (Windows CMD.EXE). When you type a command name without giving an explicit path your shell searches each directory in the PATH list in order, looking for an executable file by that name, and the shell will run the first matching program it finds.

To check the path on Windows, type:

path

You'll get a response like:

C:\;C:\WINDOWS;C:\WINDOWS\COMMAND;


To add Perl to the path, type:

path %PATH%;c:\perl\bin


To check the path on UNIX-y systems, type:

echo $PATH

You'll get a response like:

/bin:/usr/bin:

And to add Perl to the path:

  csh users type something like:
  setenv PATH $PATH:/usr/bin

  sh/bash/ksh users type something like:
  PATH=$PATH:/usr/bin
  export PATH

Syntax Errors

perl -c test.pl
String found where operator expected at test.pl line 1, 
near "printt "Hello World!""
(Do you need to predeclare printt?)
syntax error at test.pl line 1, near "printt "Hello World!""
test.pl had compilation errors.

This is a simple syntax error typo problem. Perl can not find the function 'printt'. You need to fix all these.

Missing Modules

Modules are Perl's versions of libraries. There are modules available to do everything from read in CGI data to allowing you to write your script in Latin! To use them they need to be installed.

perl -c test.pl
Can't locate Some/Module.pm in @INC (@INC contains: 
C:/Perl/lib C:/Perl/site/lib .) at test.pl line 1.
BEGIN failed--compilation aborted at test.pl line 1.

This is telling you that the module Some::Module required by a 'use Some::Module;" directive is not installed. You will probably need to install the module. To install a module see A detailed How-To for locally installing modules and Installing modules on Win32. You may be able to get away with commenting out the use Some::Module; directive to test compile you script but this will fail if you have code like print Some::Module->Cool_Function; later on.

2. Are you using strict and warnings?

#!/usr/bin/perl -w # "-w" turns on all sorts of warnings about probable errors. 
use diagnostics;   # optional; causes warnings to be explained in greater detail.
use strict;        # generates compile and run-time errors for certain unsafe constructs.

While use strict and warnings are not required for any perl script to run (including CGI) they can help you sort out all manner of problems and are highly recommended. See: Use strict warnings and diagnostics or die

3. Are you using CGI.pm to parse your CGI input data?

The leading block of reusable code for CGI purposes is CGI.pm. In a nutshell there is a lot more to parsing CGI data than first meets the eye - both from the functionality and security point of view. If you can't give ten good reasons not to use CGI.pm then *use it*. It is part of the standard perl distribution so should be available on any system that has Perl. Using it to get your form parameters is as easy as:

use CGI;            # use CGI.pm
my $q = new CGI;    # create new CGI object

# now get value of a form parameter named 'name'
my $name     = $q->param('name');

# now get an array of values of all form parameters named 'option'
my @options  = $q->param('option');

As you can see it is very easy to use CGI.pm. Besides this tutorial good entry points to all things CGI.pm, and why you should be using it are: use CGI or die; and No excuses about not using CGI.pm.

Uploading to the Server

1. Did you upload in ASCII mode?

Assuming all has gone to plan so far we are ready to upload the script to the server. The server is the computer that is actually connected to the WWW and whose job it is the serve up HTML documents and process CGI scripts. A Perl script is just a text file. Unfortunately different computer operating systems handle text files differently. On UNIX-like systems, the line ending is a linefeed character (\n). On Windows, it is a carriage return + linefeed sequence (\r\n). On Macintosh, it is a carriage return character (\r). Clearly, some conversion will be in order when sending files from one system type to another. Never fear — the problem has been solved for you. All you need to do is to make the transfer (usually by FTP) in ASCII mode. In this mode the line endings will be automatically converted for you.

Where you can strike problems is when you transfer a file in *BINARY* mode. In this mode the file is sent exactly as is so line endings are not converted. This can and does lead to problems. Make sure you transfer in ASCII mode.

2. Did you upload to the correct directory?

Most servers have a specific directory used to store Perl CGI scripts. Typically this will be something like:

/www.mydomain.com/cgi-bin/

If you're not sure, ask your system administrator about it. Whatever the correct directory is, that is where you will want to upload your scripts into.

Setting Up on the Server

OK so now we have our script in the cgi-bin. We need to set it up so that it can be executed by the server when it is called. Here is how CGI works in a nutshell:

(i) Browser requests a resource via a URL that is aimed at a CGI script, say via a link like:
  <a href="http://www.mydomain.com/cgi-bin/myscript.cgi">Run my script</a>
 
(ii) The server receives the request and goes "Ah ha! This is a cgi script that I need to run!"

(iii) The server executes the script passing it data via the environment variables in the $ENV hash and via STDIN. The script can access this data and process it. The script generates output on STDOUT

(iv) The server takes the output of the CGI script from STDOUT and returns it to the browser pretending it received a request for a static HTML document.

We can have problems in any of these areas. Of course!

1. Is the HTML document that calls the script pointed at the correct place?

In order to execute the script you need to ask for it to be executed. If you link to the URL above and your cgi-bin happens to be named something else (like, say, localcgi) then you will get a "404 Document not found" error. Of course the server can't find the document in /www.mydomain.com/cgi-bin/! It is somewhere else.

2. Is the server configured to execute the script, and are the permissions set?

You can't execute an HTML document, well at least Perl can't compile it! You also don't want to return the code in your script to the browser when it is requested. So it is obvious that the server needs some way of deciding if it needs to return the requested document of execute it. As you might expect there are many ways to achieve this result. Your systems administrator should know the exact details for your system but here is a general guide.

In UNIX, all files have permissions. They have permissions for the owner, the group, and everybody else. The permissions are 4=read, 2=write, 1=execute. Thus a permission of 7 means that the file can be read , written and executed as 4+2+1=7. A permission of 5 means the file can be read and executed as 4+1=5. As a general rule you will want permissions of 755 for your file which grants you read, write and execute permission and everyone else read and execute permission. You set this using the command chmod 755. Directories also have permissions which need to be correct but they should be fine if this is a standard cgi-bin.

Some servers are configured to recognize any file that ends in .pl, .pm, or .cgi as a Perl executable (especially under Windows) and act accordingly. You will need to use one of these extensions if that is the case so the server knows to execute the file. Still other servers assume that all files in certain directories (ie cgi-bin) are executable. The bottom line is is all else fails then check with your sysadmin.

3. Is the shebang line pointed at the Perl interpreter?

In UNIX, if a request for a file marked as executable is made, then the first 2 bytes are examined for the #! shebang sequence. If this is found then the shell executes that file using the executable file it finds at the path following the #!. Therefore, what the line

#!/usr/bin/perl

actually says is: "Dear shell, if this file is called and it has read and execute permission please execute it using the executable file /usr/bin/perl". If the perl executable is not in the /usr/bin/ directory, the shell will complain about not being able to find perl. Unless the script has permissions of at least 5 (read and execute) the shell/Perl will not be able to read it (as it must) in order to execute it (as we want).

Some servers are configured not to allow the execution of CGI scripts for security reasons, once again talk to ye sysadmin.

Debugging

1. Before doing anything else add this code to the top of your script.

If you add this code your script will run for long enough that any errors appear in the browser window making it easy to see the problem, presuming of course that you have followed the advice above! This will avoid you getting the less than useful 500 Internal Server Error or Premature end of script headers/Malformed script headers messages. You add this code just below the shebang line and before everything else. That's everything else. The reason for this is to minimise the lines which can cause problems before we reliably direct output (including syntax errors) to the browser

#!/usr/bin/perl -wT

# ensure all fatals go to browser during debugging and set-up
# comment this BEGIN block out on production code for security
BEGIN {
    $|=1;
    print "Content-type: text/html\n\n";
    use CGI::Carp('fatalsToBrowser');
}

# all the rest of the code goes here

Because this code is in a BEGIN block it is executed before everything else. Even before most of the script is compiled. In it we do three vital things to ensure that all errors from this point on will appear in the browser window thus making debugging *much* easier. You could look in the server logs to get the same information but as you may or may not have access to them and they can be anywhere on the server it is easier to use this instead.

The $|=1; forces buffer flushing. This ensures that output from the script goes immediately to STDOUT. Trust us. But if you really want the gory details, read Suffering from Buffering.

Next we print a complete valid header. In the HyperText Transport Protocol (the http bit in http://www.mydomain.com) every request and response needs a valid header. It may or may not include a body but it must have a header. The end of the header is recognised by the \n\n sequence. This prints one blank line. If, for any reason, a blank line is printed *before* the "Content-type: text/html\n\n" you will get a premature end of script header error. By printing a valid header we avoid this. The code here assumes that you want to output HTML, but it's also possible for a CGI script to output plain text, an image (JPEG, PNG, etc.), PDF, Microsoft Excel format, or any of a virtually infinite number of other file formats, using a special header which specifies data format types. The headers our script generates without this block will appear in the top left of the browser window where we can check that they are as expected. If you don't see them and your script now works your problem is that you are not outputting any valid header info.

Finally we add CGI::Carp, which essentially does the same as lines 1 and 2; but may not be as reliable as you might prefer. Combined with the two lines above virtually all errors appear in the browser window with an explanation.

While we are in this topic the -w switch on the shebang line provides heaps of free help. If you want more detail add the line use diagnostics; as shown:

#!/usr/bin/perl -wT

# comment this out on production code as it uses a lot
# of memory and you won't have any errors left anyway :-)
use diagnostics;

You may be wondering about the -T switch. Or you may not be. Regardless you want to use it. The -T switch switches on taint checking which is Perl's unique anti-cracker mode which warns you if you are doing dangerous stuff that makes it easy to hack your server via your script. See The World Wide Web Security FAQ for more details.

2. Are you checking the return values from the functions built in to perl?

Most of the file and system functions set $! and have return values that you can test thus:

open(FILE, "</file.txt") or die "Error opening /file.txt Perl says: $!\n";

$! will contain an error message that will give you more information on where your program is going wrong. The perlfunc documentation will give you more information on the return values from functions. It is very handy to know that the reason your script is not working is because you can not find/open/write to a file for instance.

3. Are the modules required by your script actually installed on the server?

As we saw earlier if you script uses modules they need to be installed. Some modules come with perl (like CGI.pm) but others need to be installed. If you have followed the advice so far you will get the typical:

Can't locate Some/Module.pm in @INC (@INC contains: 
C:/Perl/lib C:/Perl/site/lib .) at test.pl line 12.

error message in your browser window. Get you sysadmin to install them. Or see the installing modules links above.

4. Don't be afraid to Ask

If you have tried all the stuff above and have an error you can't fix, ask the Perl Monks. Many Monks do this stuff for a living so can probably help. In fact, you will often find that if you show you can help yourself, people will fall over themselves trying to help you. Good luck and enjoy the journey.

Credits

This tutorial, Getting Your CGI Scripts Working, was originally written by tachyon (Dr James Freeman). Further improvements were made or suggested by:

It is now collaboratively maintained by the PerlMonks Pedagogues.