dictionary of textFiles

Scenario: suppose you drop into a project ten “books”, i.e. ten textFiles where each textFile correspond to a book.
BrothersKaramazof
WarAndPeace
etc

Since the real project contains about one hundred of such (smaller!) files, I thought of creating a dictionary (setBookList):
Meantime I have prepared a endofLine-separated string constant (myBooksList) containing the book reference and the name of the book, as below:
WaP, WarAndPeace
bK,BrothersKaramazof
etc.

setBookList = new Dictionary
dim tempBk as String = ReplaceLineEndings(myBooksList,EndOfLine)
dim mTempBook() as String = splitB(tempBk, EndOfLine)
for i = 0 to mTempBook.Ubound
dim subSp() as String = split(mTempBook(i), “,”)
setBookList.Value(subSp(0)) = subSp(1)
next

Now, when I want to retrieve the book I send to a function the bookReference in order to retrieve the book:
if setBookList.HasKey(mTitle) then
Return setBookList.Value(mTitle)
end if
Return “”

Problem is that I dont get the bookFile stored in the project, but a simple string, for instance the string WarAndPeace.
I guess I pretty well understand the reason (I passed a string); yet I dont know how to proceed in order to get the text of the file WarAndPeace.

Any idea how to get the “book” itself? Thanks.

BTW, I thought of dumping each “book” into its own constant, or to make a database of them, but I would prefer keeping the files as files, since sometime I edit them and opening the project I dont need to modify the constants.

You might be better off storing the title and/or path to a file, and NOT the entire book.

a) up front time to load is much longer, as you are loading “books” that you don’t even know that the user will access
b) memory overhead is MUCH larger (uh… War and Piece is no “dime” novel)

by storing the title/path, you can “load on demand”
a) accesses only the books the user wants, when they want it
b) minimal overhead, as only the current “book” is in memory

Here’s how I’d do this (probably)…

Store all the files in a folder and name each file as “ref - title.txt”. Set up a “copy” build script that copies the entire folder into the App’s Resources folder.

When the app starts, build your Dictionary (or a temp SQLite database) by walking through that file and extracting the name of each .txt file in that folder. Set the Dictionary key to the ref and the value to a Pair of Title : FolderItem.SaveInfo (or set each row of your SQLite table to reference, title, contents).

When you want to retrieve the contents of the book, you can extract its title and recreate the FolderItem with the SaveInfo, or extract the contents directly from the database.

After that, editing a book or adding a new one becomes a trivial matter as long as you stick to the naming convention.

First, the textFiles are not as huge as Brothers karamazof; that was only an instance to make myself understood. The average size of the files is 60K.
But you certainly have a point.
On the other end I need all the files to be loaded, since the app works like a database where the user can search for words or sentences contained in any of the books.

Oh, then you should absolutely go with a SQLite database. Creating the database at each startup should be trivial. You can even keep it in your App Data folder and include a field for modification_date. On startup, you will only need to insert new records or update ones where the modification date has changed. (Or delete ones that are no longer present, if that’s an issue.)

Later, you can construct SQL to search the database as desired.

Then perhaps rethink your approach… and create something that breaks the books down, indexes the data and stores in a database… (think GOOGLE’ish)

or search the entire book “string” using SQL LIKE operator… but that will not be fast (but about as fast as searching thru a dictionary of string)

Ken, I’ll try to build the dictionary as you suggested.

Meanwhile, the way I have been addressing the matter is thru a function, where I feed the ref string and get the related file:
myBook = returnBook(ref)
Function returnBook(ref as string) as string
select case ref
case “aa”
return fileA
case “ab”
return fileAB
etc.

Where fileA and fileAB are not strings, but the files htemselves. The reason I thought of switching to a dictionary was, may be, a psychological doubt that the function was not so elegant as using a dictionary.
What do you suggest? Keeping my old function?
As I explained in my last post, I actually need all the files for searching.

Dave: may be I’ll have to take that step.

Yes, I’ve been thinking of that too.

Create a class that represents the book. Properties in the class will be Title, Reference, Contents. Change the function so it accesses the database and returns in instance of that class instead. Create a second function that takes keywords instead of a reference and returns an array of that class. That function will search the database using the keywords and construct the array.

Just some ideas…

Fine, I got the drift. Thanks.

[quote=205081:@Carlo Rubini]First, the textFiles are not as huge as Brothers karamazof; that was only an instance to make myself understood. The average size of the files is 60K.
But you certainly have a point.
On the other end I need all the files to be loaded, since the app works like a database where the user can search for words or sentences contained in any of the books.[/quote]
Prebuild the database BEFORE shipping your app
Even if you need to add files after the initial set up you can do that in code
I’d leave the files as separate files and just build a full text index in the sqlitedatabase
Then you can do all kinds of searches on the full text index abut NOT have to actually have the text in the db (since you dont actually search that)

I’m working on prebuilding the database.