Dealing With Duplicate Data From Form Submissions

    August 14, 2003

We’ve received several requests for a method of verifying data submitted with a web page form is unique. So I made a kit that can be used to implement such a method, and give it to you in this article.

The method compares a form item (email, name, whatever) with those previously submitted. Here are a few reasons for using the method:

–In a drawing, ensure the same email address can’t be entered twice.

–Let the form user know when a requested user name has already been taken.

–Identify those who use your web site’s feedback system more than once as special people.

How To Do It

We’ll use the freely available Master Feedback from for an example implementation. Do it with that script first to get a bit of practice and to see how it works, then modify your own script.

Master Feedback’s function is to send an email to you with the form submission information whenever someone uses your feedback form. We’ll modify Master Feedback so it will keep a database of user’s email addresses. Then, whenever the form is used, your email will also tell you whether or not the user’s email address was previously submitted.

Grab a copy of Master Feedback at the above URL. You’ll be doing two things to the script:

1. Inserting a subroutine. This subroutine does all the work of maintaining the database and determining whether or not the same email address was previously submitted.

2. Inserting code to tell the script to use the subroutine, along with instructions what to do depending on whether or not the email was a duplicate.

Inserting the Subroutine

The subroutine can be inserted anywhere in your script so long as it’s below the first line, not within another subroutine, and above any lines composed of:


For Master Feedback, insert the subroutine at the bottom of the script.

The subroutine will create its database file if it doesn’t already exist. (If it doesn’t automatically create the database file, ensure the directory where the database will exist has write permissions.)

On line four of the subroutine, between apostrophes, you can specify the file name of the database the script shall use. If you want to put the database into its own subdirectory, specify the path as well as the file name — the subdirectory must exist and have write permissions. Here is the subroutine:


When the subroutine is called (see “Telling the Script to Use the Subroutine,” below), it receives an information item to compare to the data already in the database.

The script first checks whether the database file is locked. It has to do this because several copies of the script with the subroutine could be running simultaneously. Allowing two or more simultaneous accesses to the database could corrupt the contents.

The subroutine waits up to 12 seconds for any existing lock to go away (to be removed by another script that had been using the file). After that, it assumes the lock was left inadvertently and proceeds.

The subroutine:

1. Locks the database so other scripts stay out. (This lock only works with scripts that use the same locking mechanism.)

2. Reads the entire database.

3. Compares the item it received when it was called with the items in the database. This is a case-insensitive compare. (“eMail@doMain.coM” is the same as “”)

4. If the item it received when it was called does not already exist in the database, the subroutine adds the new item to the database.

5. Unlocks the database.

6. Returns the digit 0 if the item it received did not already exist in the database. Otherwise, returns the digit 1.

The code that called the subroutine then decides what to do depending on whether the subroutine returns 0 or 1.

For Case-Sensitive Comparison —

If you want to have a case-sensitive comparison (“eMail@doMain.coM” is not the same as “”), then find this line just below the half-way point of the subroutine:


Remove the “lc” from the line in both places, and change the comment line to reflect the new actuality. (In Perl, “lc” is a function to turn something into all lower-case.)

You’ll then have:


Telling the Script to Use the Subroutine

The location in the script where you place the code to call the subroutine depends on the script you’re modifying and the purpose for using the subroutine.

If you want to abort the normal operation of the script should the subroutine return the digit 1, you would insert the code early on. If you want to let the script run its course, only modifying it’s operation slightly depending on the digit that is returned, insert the code where you want the modified operation to happen.

For Master Feedback, we want the script to run its course. But we also want to have the script make a notation in the email we receive concerning whether the email address submitted with the form is unique or is a duplicate of one previously submitted. We’ll have the script print the information at the bottom of the email, which means inserting the code immediately above the line that closes the email the script is sending.

Simply copy the code provided below and paste it into the Master Feedback script right above line 123 (if you have version 2.5). The line is:


If you do not have version 2.5, either pick it up at the URL near the beginning of this article or locate the appropriate line in the version you’re using.

Note: The modification for version 2.5 was recent. It addressed a new spammer exploit that’s been used on other scripts (in addition to the widely known FormMail exploit), a method of slyly inserting Cc: and Bcc: email addresses in the place where a web page form asks for an email subject or other information used in the outgoing email header.

Once you’ve found line 123 (or in another version, the line that closes the MAIL), immediately above that line insert:


The above code will add a blank line to the bottom of the email sent to you, the web master, and either the words “_________ is a duplicate!” or “Unique Addy: _________” (with _________ being the email address submitted on the feedback form).

To Check Form Items Other Than Email —

In the first line of the above code, replace the ’email’ in $In{’email’} with the form field name you want to check. If you want to check the realname form field, use $In{‘realname’}. If you want to check a custom form field, use $In{‘Custom Field Name’}. (Replace ‘Custom Field Name’ with the form field name you want to check.)

Any form field can be checked that way.

Checking More Than One Form Field —

To check more than one form fields, use only one instance of the subroutine but launch the subroutine more than once.



The above will check both the ’email’ and ‘realname’ fields of your Master Feedback form submissions.

Modifying Other Scripts

Other Perl scripts can be modified in a similar fashion.

Let’s take Master Form from as another example. Let’s suppose we want to modify Master Form so it uses one email template when email addresses are first used and a different email template whenever an email address is used a second or subsequent time.

First, put the subroutine at the bottom of the script.

Then, at about line 36, you’ll see:


The above code will cause Master Form to use the appropriate template file when it composes the emails it sends out. (The comma in front of the template location is to separate that data from any other email templates that might have been specified in the hidden fields of the form.)

If you want to serve a different thankyou page depending on whether or not the email address was unique, use the lines (for personalized thankyou page):


The lines can be added immediately above or below the mail template lines, if you want to specify both mail template and thankyou page.

As you can see, that one subroutine can do a lot of work. With the examples, you now know how to modify your own scripts for your own purposes.

“WillMaster Possibilities” ezine