[#CACHE-249] Performance improvment for large disk persistence usage

OSCache

Performance improvment for large disk persistence usage

Created: 24/Jul/06 03:40 PM Updated: 24/Mar/07 06:19 AM

Component/s:

Persistence

Affects Version/s:

None

Fix Version/s:

2.4

File Attachments:	1. HashedDirectoryDiskPersistanceListener.java (0.6 kb)
Environment:	UFS filesystems

Description

« Hide

On UFS filesystems there is a limit of 10.000 files per directory. Directory lookup is done using sequential searching. Large directories are
likely to have poor performance because of this.
A filesystem is not a database. If you *have* to use it as one, try an extra level of indexing.

So, a 'leveled' implementation for disk storage is needed. Like: http://squid.visolve.com/squid/squid30/logs.html#cache_dir

All

Comments

Change History

Sort Order:

[ Permlink | « Hide ]

Sander de Boer - [12/Jan/07 09:44 AM ]

added a fix to resolve this problem

[ Permlink | « Hide ]

Sander de Boer - [12/Jan/07 09:55 AM ]

this fix stores a cache file in a 3 level directory structure
eg.
0396AABAD3465158.cache
is stored in:
/0/3/9/0396AABAD3465158.cache

[ Permlink | « Hide ]

Lars Torunski - [28/Jan/07 06:36 AM ]

I prefer to patch the HashDiskPersistenceListener and not to introduce a new class.

[ Permlink | « Hide ]

Lars Torunski - [17/Feb/07 06:40 AM ]

The HashDiskPersistenceListener class was changed and the performance improvement is available by the current SVN revision 402.