History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: CACHE-184
Type: Bug Bug
Status: Closed Closed
Resolution: Duplicate
Priority: Critical Critical
Assignee: Andres March
Reporter: Simone Avogadro
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
OSCache

Filter deadlock with external apps (mostly spiders)

Created: 06/Jul/05 09:05 AM   Updated: 07/Jul/05 05:09 AM
Component/s: Filters
Affects Version/s: 2.1
Fix Version/s: 2.2 RC

Issue Links:
Duplicate
This issue duplicates:
CACHE-170 Data race handling Cache.updateStates... Critical Closed
 

Flags: Important


 Description  « Hide
We found a very bad issue which causes the cache to deadlock and completly block access to some resources.
The problem is related to how cache keys are updated,
currently OS-Cache will mark a cache-key as being 'created/updated' and will block all forthcoming requests to that resource until the first request is completed

we however observed the following behaviour:
- Client Thread 1: GET /upload/problem.pdf (which is 200K)
- Client Thread 1: READ 200 Bytes
- OS-Cache Thread 1 (serving Client Thread 1): create cache entry and block resource /upload/problem.pdf
- Client Thread 1: decide that this may be a PDF, spawn Client Thread 2 to try indexing, wait for Thread 2 to complete
- Client Thread 2: GET /upload/problem.pdf
- Client Thread 2: READ 1000 Bytes
- OS-Cache Thread 2 (serving Client Thread 2): wait for OS-Cache Thread 1 to complete

as a result now no one will bne able to access that resource and we'll have lost 2 threads/sockets.
This issue definitively needs to be fixed, owr solution is as follows:

class: com.opensympony.oscache.base.Cache:

/**
     * Simone: check if the entry is being updated
     *
     * @param key
     * @return
     */
    public boolean isUpdatingEntry(String key) {
        EntryUpdateState updateState = getUpdateState(key);
        if (updateState==null)
            return false;
        synchronized (updateState) {
            if (updateState.isUpdating())
                return true;
            else
                return false;
                
}
    }


class: Filter:

added the following code before starting to cache /acntuall within the try block):
            // avoid blocking when 2 clients
            if (cache.isUpdatingEntry(key)) {
                // 1: try delaying the request to allow the other thread to complete
                try {
Thread.sleep(5000 +(System.currentTimeMillis()%4000) );
                } catch (InterruptedException ie) {
                    // that's ok
                }
                
// 2: is still busy bypass the cache
                if (cache.isUpdatingEntry(key)) {
chain.doFilter(request,response);
return;
                }
            }


 All   Comments   Change History      Sort Order:
Lars Torunski - [06/Jul/05 03:55 PM ]
Can you please compare this issue with CACHE-170, hence we can close this issue as a duplicate.

Andres March - [06/Jul/05 06:07 PM ]
dup of CACHE-170

Simone Avogadro - [07/Jul/05 05:09 AM ]
 Sorry but I'm not sure if CACHE-170 fixes my problem, when cache is configured to be blocking and the 'generating' process does not request more bytes then it's not a race condition internal to OS-Cache, but rather a distributed deadlock between the client (which is neither disconnecting nor continuing to get) and the cache. This problem can obviously occur only with the filter.
 We cannot accept a fully non-blocking solution because there are conditions (e.g.: server outage and restore) which will cause floods of accesses, under these circumstances we would need to keep other clients on wait for some seconds before trying to generate, but should _not_ block indefinitely because of the above considerations.
 But maybe I simply miss some configuration params which would allow me to put a timeout to the blocking cache algorithm.