Sunday, 15 June 2014

python - Celery worker hangs on ZEO database access (race condition?) -


problem

Sealier workers hanging on performance when using a package accessing a JEEO server However, if I had to bring the server directly within tasks.py , then there is no problem.

Background

I have a program that reads and writes to file, because I want many users to integrate this database together and modify it , I have managed it, which should secure it in many processes and threads. I define the database within a module of my program: ZOO import import from ZOD.DB Import DB addr = 'localhost', 8090 storage = ClientStorage.ClientStorage (addr, wait = false) to import clientstorage. DB = DB (storage)

SSCCE

I'm clearly trying more complex operations, but suppose I only have one root object keys Want, or its children I can cause problems in this context.

I create a dummy_package with the above code in a module, databases , and a bare bones module is used to access the database:

  # main.py def get_keys (dict_like): return dict_like.keys ()   

If I do not have dummy_package , I can import the database and use the root without any problems:

  #work.py from dummy_package Import data @ Task () def Simple_task (): connection = databases.db.open () key = connection.root (). Key () connection.close ();   

However, trying to pass a connection or a code to a child route works indefinitely.

  @task () def simple_task (): connection = databases.db.open () root = connection.root () ret = main.get_keys (root) # hang indefinitely ... < / Code>  

If this difference makes any difference, then these celery functions are used by the Django.

Question

So, first of all, what's going on here? Is there some kind of race situation due to reaching the ZEO server like this?

I could all the databases to make the responsibility of Salary, but it would be for the ugly code. Apart from this, it will ruin my program's ability to work as a standalone program. Is it not possible for a sealer to interact with ZEO in a normal routine?

Open connection or its root object as a global one.

You need connection per-thread; Just because Zoe makes it possible to access many sources, it seems that you are using something that is not thread-local (like a module-level global in databases).

Save DB globally, but call db.open () during each task.

No comments:

Post a Comment