.. index::
   single: web page; dynamic

CGI - Dynamic Web Pages
=======================

*CGI* stands for *Common Gateway Interface*. This interface is used by
web servers to process information requests supplied by a browser.
Python has modules to allow programs to do this work. The
convention used by many servers is to have the server programs that
satisfy this interface end in '.cgi'. That is the convention used
below. All files below ending in '.cgi' are CGI programs on a web
server, and in this chapter, they will all be Python programs
(though there are many other languages in use for this purpose).
These programs are often called scripts, so we will be dealing with
*Python CGI scripts*. 

*You cannot run a .cgi file from inside Idle.*

.. _CGI-Example:

An Example in Operation
-----------------------


.. index::
   single: web page; execution sequence

The web examples folder provides a simple web server, built into Python, that you can
run on your own computer.  (It is also possible to set your computer up 
with the right software to be a server for the Internet - 
that is *totally unnecessary* for this class.)

Windows  
  In an operating system file window, go to the folder with the www
  examples. Depending on the setup of your operating system,
  there are several ways to start the local server that might work.
  
  #. Double click on ``startServer.cmd``, 
     which I have placed in the example www folder. 
     If this does not work, try:
  #. Right click on ``localCGIServer.py`` in the File Explorer window,
     and select Open With -> Python Launcher
  #. If neither work, check if you need to 
     modify your Python installation,
     covered in :doc:`pythononwindows`, and then try the 
     ``startServer.cmd`` approach again.

Mac
    This is more involved the first time.  See :doc:`pythononmac`.
     
You should see a console window pop up,
saying "Localhost CGI server started". 
This approach starts the ``localCGIServer.py`` server without monopolizing Idle.
Once the server is
started, *leave the server console window there*
as long as you want the local server running for *that folder*. 

.. warning::
   Do *not* start the local server running from inside Idle.
   It will monopolize Idle.

.. note::
   If the server aborts and gives an error message about
   *spaces in the path*, look at the path through the parent directories
   over this www directory. If any of the directory names have spaces
   in them, the local file server will not work.

In case of this error, either go up the
directory chain and alter the directory names to eliminate spaces
or move the examples directory to a directory that does not have
this issue. 


For a very simple but complete example:

#. Make sure you have the local server going.
#. Open the web link http://localhost:8080/adder.html 
   (preferably in a new window, separate from this tutorial).
#. You see a web *form*. Follow the instructions, enter integers, and click
   on the :guilabel:`Find Sum` button. 
   You get back a web page that obviously used *your* data.  
#. Look at the local server console window. You should see a log of the activity
   with the server. 

We will end up completely explaining the web pages and .cgi file
needed to accomplish this, allowing you to generalize the idea,
but for now just see how it works.

First consider the rather involved basic execution steps behind the scene:

#. The data you type is handled directly by the browser. It
   recognizes forms.

#. An action instruction is stored in the form saying what to do
   when you press a button indicating you are ready to process the
   data (the :guilabel:`Find Sum` button in this case).

#. In the cases we consider in this tutorial, the action is given
   as a web resource, giving the location of a CGI script on some
   server (in our cases, the same directory on the server as the
   current web page).  It is a resource handled by the local server,
   when the URL starts with "http://localhost:8080/" followed by the
   name of the starting web file.  *All the URL's you use*
   *for this section and its exercises should start that way*.

#. When you press the button, the browser sends the data that you
   entered to that web location (in this case ``adder.cgi``, in the same
   folder as the original web page).

#. The server recognizes the web resource as an executable script,
   sees that it is a Python program, and executes it with the Python interpreter, using the data
   sent along from the browser form as input.

#. The script runs, manipulates its input data into some results,
   and puts those results into the text of a web page that is the
   output of the program via ``print`` statements.

#. The server captures this output from the program and sends it
   back to your browser as a new page to display.

#. You see the results in your browser.


*Close* the server window.

Test what happens if you 
try to reload the web link http://localhost:8080/adder.html. You should
get an error, since you refer to localhost, but you 
*just stopped the local server*.

For the rest of this chapter, we will be wanting to use the local
server, so *restart* it
in the example www folder,
in a manner appropriate for your operating system:

* Windows: whatever worked when you started it the first time.
* Mac: Start the local server whatever way worked last time,
  either double clicking on ``cgiServerScript`` that you should have created,
  or
  right/control click on ``localCGIServer.py`` in the Finder window,
  and select Open With -> Python Launcher

Now you can *keep the local server going* 
as long as you want to run CGI scripts from the same folder.  

If you ever want be have cgi scripts and supporting files
in a different folder, *stop the server for any other folder* first,
and start it up in the folder where you have your materials. 

A Simple Buildup
----------------

Before we get too complicated, consider the source code of a couple
of even simpler examples.

.. rubric:: hellotxt.cgi

.. index::
   single: #!

The simplest case is a CGI script with no input that just
generates plain text, rather than HTML.
*Assuming you have your local server going*, you can go to the link
http://localhost:8080/hellotxt.cgi. The code is
in the www example directory, ``hellotxt.cgi``, and below for you
to *read*:

.. literalinclude:: ../examples/www/hellotxt.cgi

The top line is what tells the operating system that this is a Python 3
program. It says where to find the right Python interpreter to process
the rest of the script. This exact location is significant 
on a Unix derived server (like any Mac with OS X). 
In Windows the only thing important in the line is the distinction
between Python 2 and 3.
If you leave the line there
as a part of your standard text, you have one less thing to think
about when uploading to a Unix server or running on a Mac.

The first print function is telling the server receiving this
output that the format of the rest of the output will be plain
text. This information gets passed back to the browser later. This
line should be included exactly as stated IF you only want the
output to be plain text (the simplest case, but *not our usual*
case).

The rest of the output (in this case just from one print function)
becomes the body of the plain text document you see on your browser
screen, verbatim since it is plain text. The server captures this
output and redirects it to your browser.

.. rubric:: hellohtml.cgi

We can make some variation and display an already determined *html*
page rather than plain text. Try the link
http://localhost:8080/hellohtml.cgi. The code is in the www example
directory, ``hellohtml.cgi``, and below for you to *read*:

.. literalinclude:: ../examples/www/hellohtml.cgi

There are two noteworthy changes. The *first* print function call now
declares the rest of the output will be *html*. This is a
standard boilerplate line you will be using for your CGI programs. The
remaining print function call has the markup for an html page. Note that
the enclosing triple quotes work for a multi line string. Other
than as a simple illustration, this CGI script has no utility: Just
putting the contents of the last print function in a file for a
static web page ``hello.html`` would be much simpler.

.. rubric:: now.cgi

One more simple step: we can have a CGI script that generates
dynamic output by reading the clock from inside of Python: Try the
link http://localhost:8080/now.cgi. Then click the refresh button
and look again. This cannot come from a static page. The code is in
the www example directory, ``now.cgi``, and below for you to
*read*:

.. literalinclude:: ../examples/www/now.cgi

This illustrates a couple more ideas: First a library module,
``time``, is imported and used to generate the string for the
current date and time.

The web page is generated like in ``helloWeb2.py``, embedding the
dynamic data (in this case the time) into a literal web page format
string. (Note the embedded ``{timeStr}``.) Unlike ``helloWeb2.py``,
this is a CGI script so the web page contents are delivered to the
server just with a ``print`` function.

.. _adder.cgi:
    
.. rubric:: adder.cgi

It is a small further step to get to processing dynamic input. Try filling
out and submitting the adder form one more time,
http://localhost:8080/adder.html. This time notice the URL at the
top of the browser page *when the result is displayed*. You should
see something like the following (only the numbers should be the
ones *you* entered):

   http://localhost:8080/adder.cgi?x=24&y=56

This shows one
mechanism to deliver data from a web form to the CGI script that
processes it. The names x and y are used in the form (as we will
see later) and the data you entered is associated with those names.
In fact a form is not needed at all to create such an association:
If you directly go to the URLs

    http://localhost:8080/adder.cgi?x=24&y=56

or

    http://localhost:8080/adder.cgi?x=-12345678924&y=33333333333

you get arithmetic displayed without the form. This is just a new input
mechanism into the CGI script.

You have already seen a program to produce this adder page from
inside a regular Python program taking input from the keyboard. The
new CGI version, adder.cgi, only needs to make a few modifications
to accept input this way from the browser. New features are
commented in the source and discussed below. The new parts are the
``import`` statement through the ``main`` function, and the code
after the end of the ``fileToStr`` function. *Read* at least these
new parts in the source code shown below:

.. literalinclude:: ../examples/www/adder.cgi

.. index::
   cgi; module

First the overall structure of the code:


-  To handle the CGI input we import the ``cgi`` module.

-  The main body of the code is in a ``main`` method, following
   good programming practice.

-  After the definition of ``main`` come supporting functions, each
   one copied from the earlier local web page version,
   additionWeb.py.

-  At the end is the new, boilerplate cgi wrapper code for
   ``main()``. This is code that you can always just copy. I chose to
   put the initial ``print`` function here, that tells the server html
   is being produced. That mean the ``main`` method only needs to
   construct and print the *actual* html code. Also keep the final
   ``try``\ -``except`` block that catches any execution errors in the
   program and generates possibly helpful trace information that you
   can see from your browser. (How to write such error catching code in
   general is not covered in this introductory tutorial,
   but you can copy it in this situation!)

The ``main`` function has three sections, as in the local web page
version: read input (this time from the form, not the keyboard), process it, and
generate the html output.

-  Reading input: The first line of main is a standard one (for you to
   copy) that sets up an object called ``form`` that holds the CGI
   form data accompanying the web request sent by the browser. You
   access the form data with statements like the next two that have
   the pattern:

       **variable** ``= form.getfirst(`` *nameAttrib* ``,`` *default* ``)`` 

   If there is a form input field with **name** *nameAttrib*, its **value**
   from the browser data is assigned to *variable*. If no value is
   given in the browser's data for *nameAttrib*,
   *variable* is set equal to *default* instead.

   In this way
   data associated with names given by the browser can be transferred to
   your Python CGI program,
   and if there is no such name in a form feeding this cgi program, 
   the program does not immediately bomb out. 

   In this program the values associated with
   the browser-supplied names, 'x' and 'y', are extracted. I use
   slightly verbose Python variable names 
   that remind you that all values from the
   browser forms are strings.

-  The ``processInput`` function that is passed the input
   parameters from whatever source, is *exactly the same* as in
   ``additionWeb.py``, so we already know it works!

-  Output the page. In a CGI script, this is easier than with the
   local web pages: just *print* it - no need to save and separately
   display a file! The server captures the "printed" output.

This program can now serve as a template for your own CGI scripts:
The only things you need to change are 

#. The lines in ``main()`` that
   get the input from a web form, using the names from the input
   tags in the form, and assigning the string values to
   Python varibles.
#. In main call ``processInput(...)`` with all the data from the form that
   you need to pass as parameters to ``processInput``.
#. The heading of the definition of 
   ``processInput`` need to fit with the actual parameters 
   passed by the call in main().  Furthermore the ``processInput`` part can be written and
   tested earlier with a Python program in the format of the ...Web.py
   programs that we have discussed.
#.  While this is the only Python
    code, you still need to create an output web page template, 
    and
    refer to it as the parameter of ``fileToStr``.  Again, if you tested
    your logic using an earlier ...Web.py program, just use the same output web template!

A stripped down skeleton, to start a cgi program from, with comments about needed changes is in
``skeletonFor.cgi``.  If you do not want to start your code by modifying
an existing example cgi program, then you might start by copying 
``skeletonFor.cgi``.

.. note 
   The example parameters to ``processInput`` 
   are NOT directly part of the boilerplate code:
   You always need to list the actual data from
   *your* main function that you want to pass,
   when you call ``processInput`` in ``main``.
   Then in the heading of the definition of ``processInput`` you should 
   have corresponding formal parameters.

Idle and .cgi Files
    You can always start Idle a way that you have before, like on an existing .py file. 
    Then deal with editing .cgi files from inside Idle.

    For a Mac, See the Chapter 4 part of :doc:`pythononmac` for a way to start Idle on a .cgi file
    easily, after some initial work the first time.

    On Windows, the easiest thing may be to use IdleOnWindows.cmd.

    You will want to open and save .cgi files in Idle.
    Then dialog windows in both Windows and on a Mac have the same ideas,
    but different labels.  There is a file type filter field for both.
    It is labeled "Format" on a Mac. On Windows, for saving, 
    it is labeled "Save as Type", and for opening a file, there is a field
    it at the lower right, unlabeled. 

    If you want a file ending in .cgi, change the filter field to All files (*).
    For saving, be sure to enter the full file name you want, with the extension, .cgi.

    If you forget and do not change the file filter when saving, a ".py" will be added to your file name.
    Then you can rename it in an operating system file folder.


In the following
diagram, now we have discussed both the top regular Python sequence using the top, the bottom
cgi specific sequence, and the common part in the middle, as shown.  
In both cases input data gets processed into the content of a
web page that goes to a browser.  For any major application the main work
is in the processing in the middle.  Since that part is shared in both
approaches, it can be tested with a simple Python program,
before the starting and ending steps
of the input and output flow are changed for the cgi client/server model.

.. figure:: images/dynamicWebFlow.*
   :align: center
   :alt: image
   :width: 461.25 pt

The only part that still needs details explained is for web forms.
Before going on to that, it is time to talk about handling errors
in the CGI scripts we have been discussing.

The programs discussed here follow the simple pattern: 

   get all input data; process input data; output results all together. 

This pattern can be used both in a testing ``...Web.py`` version *and*
in a dynamic cgi version, using a ``processInput``  function common to both.

Clearly more complicated patterns are often needed 
when input, processing, and output are mixed in a different way, 
but for these examples and
for the chapter exercises, this is all we need.


.. index::
   single: cgi; errors

.. _Errors-in-CGI:


Errors in CGI Scripts
---------------------

Before you start running your own CGI scripts on the local server,
it is important to understand how the different kinds of errors that
you might make will be handled.

If you write a regular Python program, even one that produces a web page,
you can write the code and run it in Idle, and 
*Idle will display all the kinds of errors*.

With a CGI script, you can still use Idle to write your code,
but *you cannot run the cgi code in Idle*, and errors show up in *three*
different places, depending on their type:

Syntax errors
    You are encouraged to check for syntax errors
    *inside* Idle, by using the menu sequence :guilabel:`Run -> Check Module`.
    If you fail to do this and
    try running a script with a syntax error, the error trace appears
    in the *console* window that appears when you start the local
    server -- better to have it fixed before then,
    while you are still editing.
    If you want an illustration, you might try changing
    adder.cgi, making an error like ``impor cgi``, and try using
    adder.html with the flawed script. (Then fix it and try again.)

    If you find it in Idle, you can jump to the line where the error was detected.


Execution Errors
    The error trace for *execution* errors is
    displayed in your *web browser*, thanks to the final boilerplate code
    with the ``try``\ -``catch`` block at the end of the CGI script.
    *If you omit that final boilerplate code, you completely lose descriptive*
    *feedback: Be sure to include that boilerplate code!* You can also
    see an illustration now. Get an error by introducing the statement::

        bad = 'a'.append('b')

    in the main function and run it.
    (Then take it out.)

Server Errors
    Your work can cause an issue inside the local server,
    not directly in the Python execution.  Some errors are
    communicated to the browser, but not necessarily all.
    Other errors appear in the log generated in the local server's window.
    You could have a file named wrong, for instance, 
    in any operating system.

    **Also, on a Mac or in Linux**, where the CGI script needs
    to be set as *executable*, an error with a non-executable CGI script
    only shows up in this log in the local server window. You should be able to fix this error by running the ``CGIfix.py`` program,
    *in the same folder*.

Logical Errors
    Since your output appears in the web browser,
    when you produced something legal but other than what you intended,
    you see it in the browser. If it is a formatting error, fix your
    output page template. If you get wrong answers, check your
    ``processInput`` function.

.. index::
   client-server testing

Here is an outline for client/server program testing,
emphasizing errors to be conscious of and avoid.  
If you have problems, please actively check out this whole list:

* You are encouraged to add your files into the examples www folder.
  If you really want to work in a different folder, 
  you will want to copy a number of support files into that folder:
  CGIfix.py and localCGIServer,py, plus dumpcgi.cgi if you want to test
  web forms separately.
  For Windows: startServer.cmd.  On a Mac be sure to run CGIfix.py,
  to generate the folder-specific version of cgiServerScript.

* If you want an easy environment to test a fancy ``processInput`` function,
  embed it
  in a regular Python program, as we have discussed for ...Web.py programs.
  Then you can test it normally in Idle.
  This will also allow you to make sure the web template, that you refer to
  in your ``processInput`` function, is in a legitimate form,
  with substitutions only for local variables.

* You can *code* a CGI script in idle, but *not run* it.
  Be sure to save it with the suffix ``.cgi``,
  *not* ``.py``.  Do *not run* it it Idle.  The only testing you can do in Idle
  is for syntax, using the menu sequence :guilabel:`Run -> Check Module` or
  its keyboard shortcut.  *Do* run that test.

* At the end of your CGI script, make sure you include the boilerplate
  code that catches execution errors.

* Remember to run CGIfix.py in the same folder as a precaution to
  clean things up, particularly with a new file on a Mac.

* Make sure your local server is going, and that all the files you
  reference are in the same folder as the local server.

* Make sure you test your page by starting it in your web browser with a
  URL starting ``http://localhost:8080/``.  If you load a web page
  directly from your file system by mistake, 
  it *will not cause an obvious error* -
  the dynamic actions will just *not* take place.  If you are not paying attention,
  this can happen and be very confusing!

* If things do not work, remember the three places to check for errors.
  After initially checking for syntax errors in Idle,
  the remaining errors might be displayed on the output page in the browser
  (if you confirmed that you have the try-catch boilerplate code at the end).
  If that does not totally explain things, 
  remember to check the server console window, too.

We have not covered web forms yet, but rather than bite off too
much at once, this is a good time to write your own first CGI
script in the following exercise.

.. _quotient.cgi:

Quotient.cgi Exercise
~~~~~~~~~~~~~~~~~~~~~

\* Modify :ref:`quotientWeb` and save it as a CGI script
``quotient.cgi`` in the *same* directory where you have
``localCGIServer.py`` and your output html page template from ``quotientWeb.py``.
Make ``quotient.cgi`` take its input from a browser, rather than the
keyboard. This means merging all the standard CGI code from
``adder.cgi`` with the ``processInput`` function code from your
``quotientWeb.py``. 
Please keep the same web form data names, 'x' and 'y', as in adder.cgi, so
the main method should not need changes from adder.cgi. Remember to
test for syntax errors inside Idle, and to have the local web
server running when you run the CGI script in your browser. Since
we have not yet covered web forms, test your CGI script by entering
test data into the URL in the browser.  In particular use links
http://localhost:8080/quotient.cgi?x=24&y=56 and
http://localhost:8080/quotient.cgi?x=36&y=15. 
These URLs depend on you keeping the same data names, x and y, in the form.
After trying these
links, you can edit the numbers in the URL in the browser to see
different results.


.. index::
   single: web page; form

.. _Editing-HTML-Forms:
    
Editing HTML Forms
------------------

This section is a continuation of
:ref:`Introduction-to-Static`, intended for users running Kompozer. 
*A parallel development, without Kompozer, that may be simpler* 
is in :doc:`htmlsource`.

This section is about HTML editing using Kompozer -- not about Python. 
HTML forms will allow user-friendly data entry for Python
CGI scripts. This is the last elaboration to allow basic web
interaction: Enter data in a form, submit it, and get a processed
result back from the server.

The initial example, adder.html, had a form with only text input fields
and a submit button inside. 
*You will not need anything with further kinds of markup for the exercises.*

Now open ``adder.html`` in Kompozer. Switch to the Source view. This is
a short enough page that you should not get lost in the source
code. The raw text illustrates another feature of html: attributes.
The tag to start the form contains not only the tag code *form*,
but also several expressions that look like Python assignment
statements with string values. The names on the left-hand side of
the equal signs identify a type of *attribute*, and the string
value after the equal sign gives the corresponding string *value* for the
attribute. The tag for many kinds of input fields is ``input``.
Notice that each field includes ``name`` and ``value`` attributes.
(So there is some confusion possible here from the fact the **value**
is the name of a particular attribute, 
and all attributes have  string as their value.  I will try to use the phrase "string value" consistently for the second usage.)
See that the 'x' and 'y' that are passed in the URL by the browser
come from the **name** attribute given in the HTML code for the corresponding
fields!

Kompozer and other web editors translate your menu selections into
the raw html code with proper attribute types. This high level
editor behavior is convenient to avoid having to learn and debug
the exact right html syntax! On the other hand, using pop-up field
editing windows has the disadvantage that you can only see the
attributes of one field at a time, 
and likely need mutiple mouse clicks, plus typing. 
Particularly if you want to
modify a number of name or value attributes, it is annoying that
you need a number of mouse clicks to go from one field to the next.

If you *only* want to modify the *string values* assigned to 
existing attributes like
``name`` and ``value``, it may be easier to do in the source
window, where you can see everything at once. 
(Or use a separate html source editor, as described in
:doc:`htmlsource`.)
Making syntax errors in the html source
in not very likely if the only changes that you makle to tags
are data *inside* quoted value strings.  

The action URL is a property of the entire form. To edit it in
Kompozer via the GUI, right click inside the form, but *not* on any field
element, and select the bottom pop-up choice, Form Properties. Then
you see a window listing the Action URL and you can change the
value to the name of the CGI script that you want to receive the
form data. When you create your own web form, I suggest you make
the initial action URL be dumpcgi.cgi. This will allow you to debug
your form separate from your CGI script. When you have tested that
your web form has all the right names and initial values, you can
change the action URL to your CGI script name (like quotient.cgi),
and go on to test the combination of the form and the CGI script!

If you are modifying a previous working form, it is likely easier
to use the source view, 
and just replace the value of the form's action attribute
to the new .cgi file name.

Optionally, you can test various input methods by opening and playing with
http://localhost:8080/commonFormFields.html. (Make sure your local
server is still running!)  

To allow easy concentration on the data sent by the browser, this
form connects to a simple CGI script ``dumpcgi.cgi``, that just
dumps and labels all the form data to a web page.
Press the submit button in
the form, and see the result. Back up from the output to the
previous page, the form, and change some of the data in all kinds
of fields. Submit again and see the results. Play with this until
you get the idea clearly that the form is passing on your data.

If you want to create a form and input fields *using the Kompozer GUI*,
open this same file, the www
example ``commonFormFields.html``, in *Kompozer*. The static text in
this page is set up as a tutorial on forms in Kompozer. Read the
content of the page describing how to edit 
at least forms with text input fields and submit buttons.
*The other fancier fields are optional -- not needed for the chapter exercises.*
 
Read the static text about how to edit
individual fields, and change some field parameters, save the file
and reload it in your browser, and submit again. If you change the
name or value attributes, they are immediately indicated in the
dumped output. If you change things like the text field size, it
makes a change in the way the form looks and behaves. You can
return to the original version: An extra copy is saved in
``commonFormFieldsOrig.html``.

Now we have discussed the last piece, web forms,
in the diagram for the comparison of generating web pages dynamically
by a regular Python program or a server CGI script:
    
.. figure:: images/dynamicWebFlow.*
   :align: center
   :alt: image
   :width: 490 pt

Note the last three Python videos do not directly corresponding to a single
place in the Tutorial text.  Instead they go through the entire process
for web based programs from the beginning.
Video 4.4.4b creates a ``birthday.html`` web form looking forward to
``birthday.cgi`` of video 4.4.4d.
In the middle video 4.4.4c creates ``birthdayWeb.py``, testing the process
function and output template to be used in ``birthday.cgi``.    

QuotientWeb Form Exercise
~~~~~~~~~~~~~~~~~~~~~~~~~

\* Complete the web presentation for quotient.cgi of
:ref:`quotient.cgi` by creating a web form ``quotient.html`` that
is intelligible to a user and which supplies the necessary data to
``quotient.cgi``.

Be sure to test the new form on your local server,
using the URL http://localhost:8080/quotient.html. Remember that
you must have the local server running first. You must have all the
associated files in the same directory as the server program you
are running, and you cannot just click on quotient.html in a *file*
browser. You must start it from the the URL
http://localhost:8080/quotient.html, that specifically refers to
the server localhost.

Dynamic Web Programming Exercise
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

\* Make a simple complete dynamic web presentation with a CGI
script that uses at least *three* user inputs from a form. The
simplest would be to just add three numbers instead of two. Call
your form ``dynamic.html``. Call your CGI script ``dynamic.cgi``. Call an
output template ``dynamicTemplate.html``. Remember the details listed
in the previous exercise to make the results work on localhost.
After the server is started and you have all the files, go to
http://localhost:8080/dynamic.html to test it a few times.


:ref:`websummary` starts with the overall process for
creating dynamic web pages.

.. _More-Advanced-Examples:
    
More Advanced Examples
----------------------

This is an optional section.

One of the advantages of having a program running on a public
server is that data may be stored centrally and augmented and
shared by all. In high performance sites data is typically stored
in a sophisticated database, beyond the scope of this tutorial. For
a less robust but simpler way to store data persistently, we can
use simple text files on the server.

The www example page ``namelist.html`` uses ``namelist.cgi`` to maintain a
file ``namelist.txt`` of data submitted by users of the page. You can
test the program with your local Python server. It is less
impressive when you are the only one who can make changes! 
The source code is documented for those who would like to
have a look.

You also may want to look at the source code of the utility script
you have been using, ``dumpcgi.cgi``. It uses a method of getting
values from the CGI data that has not been discussed::

    val = form.getlist(name) 

This method returns a list of values associated with a name from
the web form. The list many have, 0, 1, or many elements. It is
needed if you have a number of check boxes with the same name.
(Maybe you want a list of all the toppings someone selects for a
pizza.)

Both ``dumpcgi.cgi`` and ``namelist.html`` add an extra layer of robustness
in reflecting back arbitrary text from a user. The user's text may
include symbols used specially in html like '<'. The function
``safePlainText`` replaces reserved symbols with appropriate
alternatives.

The examples in earlier sections were designed to illustrate the
flow of data from input form to output page, but neither the html
or the data transformations have been very complicated. A more
elaborate situation is ordering pizza online, and recording the
orders for the restaurant owner. You can try
http://localhost:8080/pizza1.cgi several times and look at the
supporting example www files ``pizza1.cgi``, ``pizzaOrderTemplate1.html``,
and the simple ``pizzaReportTemplate.html``. To see the report, the
owner needs to know the special name ``owner777``. After ordering
several pizzas, enter that name and press the Submit button again.

This CGI script gets used in two ways by a regular user: initially,
when there is no order, and later to confirm an order that has been
submitted. The two situations use different logic, and the script
must distinguish what is the current use. A *hidden variable* is used
to distinguish the two cases: when pizza1.cgi is called directly
(not from a form), there is no ``pastState`` field. On the other hand
the ``pizzaOrderTemplate1.html`` includes a hidden field named
``pastState``, which is set to the value ``'order'``. (You can confirm this
by examining the end of the page in Kompozer's source mode.) The CGI
script checks the value of the field ``pastState``, and varies its
behavior based on whether the value is ``'order'`` or not.

The form in pizzaOrderTemplate1.html has radio buttons and check
boxes hard coded into it for the options, and copies of the data
are in pizza1.cgi. Keeping multiple active copies of data is not a
good idea: They can get out of sync. If you look at the source code
for pizzaOrderTemplate1.html, you see that all the entries for the
radio button and check box lines are in a similar form. In the
better version with altered files ``pizza.cgi`` and
``pizzaOrderTemplate.html`` (that appears the same to the user), the
basic data for the pizza options is only in one place in ``pizza.cgi``,
and the proper number of lines of radio buttons and check boxes
with the right data are generated *dynamically*.  To do the dynamic generation,
templates for an individual html line with a size radio button
is in the source code, and it is used repeatedly to generate multiple lines, each with
a different size and price embedded into the format string
from the program data.  These lines are joined
together and placed as one entity into the html form template. 
A similar procedure is done with the toppings and check-boxes.

The logic of using the hidden input field is futher outlined 
in :doc:`htmlsource`.

A further possible elaboration would be to also allow the
restaurant manager to edit the size, cost and available topping data online,
and store the data in a file rather than having the data hard coded in
pizza.cgi, so if the manager runs out of a topping, she can remove
it from the order form. This change would be a fairly elaborate
project compared to the earlier exercises!