Already Slipping

Lesson learned: you should never start anything on a Friday. By the time Monday rolled around I got carried away with debugging my Filament Analysis program and forgot to blog my work of the day.  So at the risk of setting a bad precedence let me recap what happened yesterday as well as what went on today.

Monday:
I am trying to get some filament results for Dave's NSF proposal since the committee like that aspect of the proposal last year.  I have rewritten my filament analysis code and just finished the bulk of the coding on Friday.  I spent all day debugging (as well as attempting to debug) my new filament analysis program. The gist of this program is that I give it the positions (ra, dec, z) of galaxy clusters then use it to estimate likely locations of filaments, specifically between close cluster pairs, similar to Mead et al. 2010).  I have to go about it this way because filaments are not nearly as over-dense as cluster and are very hard to detect where as clusters are relatively easy to detect.  I then combine the signal from each of the filaments by rotating and stacking them.  I am sure I will go into more details of this process later.  Surprisingly one of the biggest difficulties of the day was obtaining and manipulating the galaxy catalog for the field I am analyzing (a 2x2 sq. deg. area, one of the DLS fields). I am using each galaxy's full photometric redshift probability distribution, p(z) for short, so instead of a single number representing each galaxy's redshift we have 500, which results in an ~3 Gb catalog.  Most text editors choke on anything larger than 2 Gb so even mundane manipulations of this catalog (e.g. search and replace) are a challenge. vim came to the rescue though, as apparently it has no file size limitation.

Today:
More time spent debugging the code.  At face value the program is running correctly except for handeling of the masked out regions (i.e. patches where we have no data), which in some instances are causing some infinities in error estimate calculations.  I know how to fix this so hopefully the code will be up and cranking out results in time for the NSF proposal deadline (effectively at the end of this week).

On a separate note I got the referee's comments back on our submitted merging cluster paper. There were no major comments (e.g. "this work is crap you should quit science and become a preist") and only about 7 comments asking for a little more detail, which should only require some minor reanalysis.  Hopefully we can get the addressed next week after the NSF deadline.

I mentioned this blog at our Cosmology Lunch today and it made for a pretty good and discussion as well as some pretty good jokes. On the subject of openness of research and potential for being scooped the main conclusion was that it is hard enough to get people interested in you work at all so there is probably little concern for people scooping you based on what is roughly posted to a blog.

Well this is obviously too long of a post and I can't possibly keep this up so I will probably have to make following posts more concise. It will also help if I don't double up on days like this post.