methods - Repeated tasks - spawn new processes or run continuously? -

We have around 10 different Python scripts that download data from the Web, read the data from the database and that database Back in the data write in They do it repeatedly every 10 seconds (or 10 seconds after completion of the final work).

The question is, what is the best way to run these tasks? I can think of some ways:

1. There is some time that runs the work then sleeps for the interval. It can be carefully guarded like a supervisor, to ensure that it is always there.

2. By executing the script only once, and once every 10 seconds, another process.

3. Ask for a script run for 1 hour (every 10 seconds for one hour), and on keeping a watchdog, make sure the job runs once again. .

   I would like to avoid long-running procedures which actually do something because I do not want to work in memory over a long period of time.

   Additional information

   1. The scripts are different because they retrieve data from each different source, and queries, Calculate and separate data in the database.

   2. The work is done every 10 seconds, because the data is being retrieved in real-time, and we should not only update it continuously, but also keep it All historical data in the database

   3. There are lots of resources being used by scripts - MySQL connections, HTTP connections, readies connections, etc. In particular, using the first long-standing approach, we have encountered problems, MySQL connections (things like MySQL servers have gone, although all connections were closed). Therefore, leaning toward scripts to run in less time.

      What are some common approaches to this?

      Unless your script is in some way memory leak (not quite likely), they should all be identical. Therefore, for quite ease ( your time programming / debugging time is more expensive than some milliseconds at the time of machine, even every 10 seconds!) I go to a script which Checks every 10 seconds.

      OTOH looks like busy work in every 10 seconds, can not you set up so that whatever you are monitoring tells you when changes happen? Or batch records so that you can get a day's worth, can you?