Does anyone have suggestions and possible examples of loading a weblistbox with a huge amount of data from a table. I know I can load so much at a time but there is no scroll event or anything I can see with which I could load more data in while the user is scrolling down the list.
Ok…this is far and away from “Web” - but I’ve been looking for a real high performance listbox for a while. One that could handle “lots of rows” without going ape. The answer is generally “don’t do that”. My thought has always been…if I need to load up a small (in some cases 50MB) delimited file to just view a a delimited file in something besides “TextEdit” there ought to be a way to do it. What good are all these CPU cycles and gigabytes of Ram if I can’t do something so “simple” as open and view (scrub through) a large delimited file. Well, Somebody must have solved it. I have a copy of an MDB/ACCDB Viewer. http://eggerapps.at/mdbviewer I opened up a 1.77GB database file in a flash (it displays all the table names and their record counts), which is probably easy enough because I’m sure all that info is simply part of the MDB file structure. Then I asked it to open a table with 3,627,006 rows (each row has nine fields…nothing too complex…but a reasonable amount of data per record. It did so without complaining…and only took 45 seconds (during which I got a nice progress meter). The resulting scroll window worked incredibly smooth and scrolled like greased lightning. Yowza! BIg Iron? Nope… Macbook Pro 1.53 GHz Intel Core 2 Duo with 4 GB Ram. Ok…so does anyone really need 3.5 meeeelion rows in a grid? Not generally…but our tech ought to be able to handle it. I was watching the memory pressure when this large grid opened. It was “negligable”. Sure…it went up by enough where I assume they sucked the whole table into memory (about 150 MB)…but that’s what I expected. As soon as I open a different table it dropped all the memory and then grew based on the size of the new table. This puppy could load up 10,000 rows in a blink. Impressive.
With a web app, you need to consider bandwidth, unknown available RAM on the user workstation, unknown processing power on the end user workstation. These are factors that you do not need to worry about using an MSAccess viewer locally. I doubt that you could transfer 1.77GB of data over the internet to the client in 45 seconds. And if you could, the user would likely assume that there is a communication issue and quit the session in the middle of the transfer, progress indicator or not.
I still vote for transferring small chunks of data at a time.
Oh yes…for the web (which is this thread after all) I don’t recommend sending back too many rows. I built a listbox and pagination control that allows me to move back and forth though a file a configurable number of rows at a time (100, 250, 500, 1000) …or jump to any position in the rows set. The web listbox control is constrained to the browser after all. Still I’ve been a little disappointed that the performance of loading up a web listbox does not seem to be constrained by the data transport…rather by the process the framework uses to load the data into and render the control seems abnormally sluggish. That said…I compare it to the local compiled XOJO app and find the regular listbox a bit on the pokey side also…though it does offer quite a few features and it takes a while to set all those individual cell properties I’m sure. I’d be amazed if you could load up 3.5 million rows with nine columns into a XOJO listbox in a reasonable amount of time (if at all). The general consensus is that it’s “unreasonable” to do so. My thoughts… 45 seconds is not unreasonable. Usually I only want to load 10 or 20 thousand rows. That should happen “in a blink”.