Can I use Database.Batchable to read a file, process it and return an updated file?

[cayuga2]

Hi,

I'd like to process a large file.  Essentially, I need to read rows from an uploaded file, append information from Salesforce and the return the updated file.

Since the file can be arbitrarily large, I'm thinking that Database.Batchable is the way to handle this.   I'm assuming that the best way to return the file is to use HTTP to upload the result (or each batch of the result) to an external web server.

If anyone can provide hints on how to go about this, point me at sample code that I might learn from or suggest a better way to accomplish this, I'd REALLY appreciate it.

Thanks!

               Jeff

[bob_buzzard]

Database.batchable is intended to allow you to process a large number of existing records based on a query or collection you supply.  It won't let you break up an arbitrary file into batches.  Its likely you'd need to use an external tool for this, as processing large files usually breaks governor limits.

[cayuga2]

Thanks!

I was basing my approac on http://developer.financialforce.com/customizations/importing-large-csv-files-via-batch-apex/

I thought that it was a way to use the system to handle the governor limits for me.

I that won't work, and I break up the records externally, does it make sense to use Database.Batchable to do the processing?

Thanks again,

                         Jeff

[bob_buzzard]

This will work up to a point, but you still have to pull the entire file into the batch apex to allow it to be iterated.  We tried something similar but once you get files into the megabytes you quickly run out of heap space even in the batch context.  This approach may work for you depending on your file size.  

We tend to break the files up externally, carry out a number of loads to populate a staging object, then run batch apex to process these and generate the real object.

[cayuga2]

Thanks very much. I've programmed for decades, but am new to salesforce.  So, I have my code written and I'm trying to figure out how to send it a file to test.  Any suggestions on where to look?

Thanks again,

                          Jeff

[bob_buzzard]

There's a few ways to get at files.  If it were me I'd upload an attachment to a record and then process that.  There's an example of uploading attachments in VF on my blog if that helps - its a post about uploading multiples, but should be straightforward to adapt:

http://bobbuzzard.blogspot.co.uk/2011/01/uploading-multiple-attachments-via.html