Million dollars is not cool, you know what is cool? million rows
I got a mail from a user stating that opening a specific csv file crashes the CSV Viewer app. I dutifully downloaded the file and it crashed it for me as well. I checked the file size; it was slightly bigger than 50MB, "Well of-course it crashed; I specifically mentioned how to use large files with preview mode". Looks like the user did not bothered to read the application notes. I was about to reply with a link about preview mode, but i got curious: why did it crashed? My machine has 4GB; there are no other tabs open. It should be able to open 50MB file, shouldn't it?
There are two likely candidates that could be causing the crash: parsing csv or rendering of data. Generally, rendering is the heavier operation. Most of the csv viewers crash on DOM rendering with large data, but there are well-known techniques to mitigate this. Almost all rely on the fact that, at a time, only limited number of rows would be visible to the user. Some of them are
I should also mention that there are some seriously impressive libraries like Fast Grid that continue to push the limits of DOM. However at million rows DOM based approach is simply not feasible.
