Mostly this post is about “ooooh aaaaah pretty diagram”.  But below I’ll try to explain what we are doing using a web-server analogy.

The Analogy:

Imagine a farm of Google search servers: the waving wheat, the moo-ing cows.  Idyllic.  No but seriously, these server processes are getting pounded by search requests.  For the sake of discussion let’s make a few assumptions. First suppose that most requests were identical to others that occurred within the last few minutes.  For example, ever time Kobe Bryant scores a basket, a million people Google “Kobe” (and not the beef).

In such a universe, you might want to detect the onset of these correlated search waves and respond very quickly by updating a set of cached results the servers in the Western US use.  This is a tricky problem because you have thousands of machines who each see a slight uptick of “Kobe” searches and you need to make a decision to update caches that will affect those same machines, even though not one of them sees the whole picture.  And they’re all acting asynchronously.  Bummer, dude.

The Limits of the Analogy and our Solution:

Our problem, which is part of the ATLAS Trigger and Data Acquisition system, has two big differences.  Firstly, our system is partly synchronous.  This means that we have a single point, called the Central Trigger Processor, that is guaranteed to control the flow of data downstream of it (which is then asynchronously processed).  Secondly, our system really really needs the parallel processes to cooperate.  In the Kobe example you might argue that if the wave of searches was so damn big, each individual process would notice it.  Yea perhaps, its just an analogy!  In our system it takes thousands of processes working across dozens of racks to accumulate enough statistics to accurately measure these parameters of physical (and technical) interest called the Luminous Region.

We accomplished the N:N communication problem by a big fan-in and fan-out, which uses a hierarchical data-collection and -distribution.  So we aggregate all the data across the processes for about ten minutes, crunch thousands and thousands of fits on them, then make a decision.  Have the parameters we’re measuring, changed so much that we need to inform the farm?  If so we push our values to an accessible location and trigger an update.  Then WHAMO!  Those exact same processes go fetch the fruits of their collective labor.  Many hands make light work =).

In this way, we were able to coordinate so many processes without ever letting them know about each other.  And when they all do go update their values (as close to synchronicity as the farm ever gets) it still induces < 10ms of dead time on the system.  Not to shabby at all!

Well that’s all I had on the diagram.  Like I said, just an overview.  Thanks for reading!

So here’s a puzzler for today:  How can you numerically simulate two planets interacting via gravity, in a universe with infinite volume?  Of course you only have standard computing architecture available.  Once you have a guess, make gravity repulsive and see if your solution still works.

And since I’m totally an evil T.A., discuss when and how your numerical error grows “too large”.  Good hunting!

But you, my avid reader, protest “it’s a test to see how you think and reason”. Okay that’s fair, but I, for the sake of discussion, claim quicksort and other standard algorithms aren’t the right way to test an applicant’s reasoning ability. All applicants know quicksort could be asked, so we all look it up. Now, when we are sitting down during an interview, we’re trying to recall all the intricacies of a mature algorithm. Instead of reasoning through divide-and-conquer, I’m thinking “oh crap, now how to I pick the pivot to avoid those QS-attacks?”. I’m trying to remember how to get all the details right, when I should be iteratively addressing problems as they arrive.

Of course, if you’re hiring someone to squeeze out the next 1% of execution time in some DB-query, then absolutely he or she should know every damn intricacy of the standard algorithms. But for the rest of us, I think non-standard brain teasers or puzzles seem much better than reproducing a well-known mature algorithm.

IMHO =)

XKCD always getting to the heart of the matter

The trick here is to get ROOT to echo all the TBranch names to stdout (through TTree::Print()) so that we can use all our normal grep tricks to find that one name we are looking for.  Here is the script in its totality:

#!/bin/bash
#./thisScript treeName fileName
{ echo "TFile f(\"$2\", \"READ\")" ; echo "$1->Print()" ; echo ".q" ; } |\
 root -l -b |\
 grep -v "^[*]Tree" |\
 sed -n -r "s/^[^:]*:([^ ]+)\s+:.*/\1/p"
#EOF

Bash curly braces always freak me out, because they can do several different jobs. In this particular case, we are moving the output of several commands to stdin of another process. Remember in the above script, the $1 and $2 get expanded to the first and second arguments passed to the shell script. Here is, effectively, how the curly braces and ROOT call are interacting…

$> root -l -b
##ROOT ASCII ART###
>>>TFile f("$2", "READ")
>>>$1->Print()
>>>.q
$>

Which causes ROOT to vomit lots of text like this…

...
*Entries : 172375 : Total Size= 23966439 bytes File Size = 13664685 *
*Baskets : 917 : Basket Size= 32000 bytes Compression= 1.75 *
*............................................................................*
*Br 180 :mooreseg_phi : vector<double> *
*Entries : 172375 : Total Size= 23968281 bytes File Size = 8502181 *
*Baskets : 917 : Basket Size= 32000 bytes Compression= 2.82 *
*............................................................................*
*Br 181 :mooreseg_theta : vector<double> *
*Entries : 172375 : Total Size= 23970123 bytes File Size = 14002768 *
*Baskets : 917 : Basket Size= 32000 bytes Compression= 1.71 *
*............................................................................*
...

Since all that goes to stdout we can just grep/sed our way through it until we get a pretty list of branch names.  Now we just have a plain text stream.

$>./treeprint.sh CollectionTree data12_jte.root
...
mooreseg_z
mooreseg_phi
mooreseg_theta
...
$>

Pretty simple huh? How about that for “worse is better”?

I’m trying to throw up this site quickly, so there will almost certainly be bugs/spelling mistakes/swearing in posts as I go.  Be patient, and lemme know what I’ve screwed up so it can get fixed pronto.

Thanks!