Posts
1150
Comments
891
Trackbacks
1
How much of your developer career is due to Google?

I don’t know about you, but I don’t memorize syntax.  I mean, when I’m doing typical daily development work, the speed at which I can get stuff done depends pretty heavily on my short-term memory, that is, on whether I’m writing code that is similar to code I’ve been writing recently.  If it involves code of a sort I haven’t worked on in over a month, there’s a bit of friction (as the kids like to say).

Anyway, this all came to mind the past few days.  For various reasons, I had to write a fairly simple console application.  Basically, it required that I create two files for an external vendor to pick up from our FTP site.  The data for these files came from two sources: a view on top of a SQL Server database and a file produced from a 3rd party vendor application.  For the file produced from the SQL view, there was no business logic involved.  For the file produced from the vended file, there was some pretty basic business logic to aggregate data as well as a couple of other things.  Nothing complicated.

What I found vaguely interesting was how much I had to use Google to write the code.   Here’s what I had to do:

  1. For the data coming from the SQL view, I had to use ‘raw’ ADO.NET as the client did not support an ORM I could use.  I didn’t have to do too much searching here, only a little.
    1. Do you create a SqlConnection and then a SqlCommand, or the other way around?  What’s the best way of doing that using TransactionScope?
    2. Since I was using a DataReader, I remembered seeing some code from Derick Bailey about how to work around checking for DbNull and all that, so modified that slightly.
    3. Writing out a file from a List<T>, how do you do that? 
  2. For the data coming from the vended file, I had a bit more to do.
    1. I had the command that I would run from a command prompt, so how do I ‘shell out’ and run that?
    2. It turns out that the vendor application process exits before the extract file it produces showed up in the proper location.  So, how to sleep for the proper amount of time to allow the file to be produced before looking for it?
    3. A typical way of processing a file you read from disk is to process it line by line, but for various reasons, I needed instead to load the extract file into memory as a byte array, and then process it, so how do you do that?
    4. To produce the proper resulting file, I needed to calculate the last business day of the previous month.  Given that I had a source of all holidays for the year, how do I do that?
    5. Processing the extract file produced a collection of nearly a million records.  This ended up causing obvious performance issues:
      1. I started with basic LINQ to Objects code to implement the aggregation logic I needed.  Which worked, except it took nearly an hour to run, and that’s not good.
        1. Did I write the LINQ to Objects code wrong?  Have other people noted that LINQ to Objects code has performance issues?
        2. Once it was determined that operating over a collection with LINQ to Objects has issues, were there better collections to use?
        3. Given that I didn’t have a unique key that I could use per each record (eliminating a lot of options), what can I do here?  I ended up using Lookup, which reduced the processing time from almost an hour to less than a minute.

I’ve probably left out a few steps, but you get the point.  I didn’t actually need to use Google for every single step here, but you get the point. 

Setting up the solution to use the ‘DDD-Lite’ structure I prefer (console calls services which call repositories which call data access or file access) required almost no thought.  Because of my cqrs-junkie status, I actually implemented a query layer between the service layer and repository layer, but I don’t like how I did that, so I need to refactor.  I encapsulated calls to app.config with an IConfiguration interface, but I don’t like that I pass in an IConfiguration in too many layers, so I obviously did that wrong, so I need to refactor.

But all of that stuff took less time than me ‘remembering’ how to shell out to run the 3rd party vendor application, for instance, since I just haven’t done that in quite a while.

Now, admittedly, I am lazy and/or stupid.  But without Google, I would have been stuck on basic syntax issues. 

I suspect I’m not the only one who has been in a situation like this.  Or perhaps that’s wishful thinking.

posted on Thursday, April 01, 2010 6:42 PM Print
Comments
No comments posted yet.

Post Comment

Title *
Name *
Email
Url
Comment *  
Please add 3 and 3 and type the answer here: