An Engineering Perspective: Magnetic Acquires New Technology

A Q&A with Mark Weiss, Magnetic’s VP of Engineering

What are you most excited about by acquiring Adnamic?

The acquisition of Adnamic signals tremendous growth for Magnetic; however, for me, an equally exciting part of this partnership focuses on the technology. By combining Adnamic’s capabilities with Magnetic’s existing platform, we gain complete control over our product’s interaction with our audience and complete access to all of the data generated through the ad targeting process. We can now look beyond search data and gain insights into audience data collection and variables that impact real-time decision making, such as identifying the most valuable ad to show to a specific user, the optimal price, previous pages viewed, etc.

By analyzing multiple data points, we’re able to improve the performance of our product, while generating key insights that we can then pass on to our advertiser clients. By controlling every step of the advertising process, we are ultimately better equipped to deliver robust analytics and reporting, which in turn helps our advertising team optimize campaigns. Adnamic is helping us to propel forward into the real of heart of the data – analytics. It will be a fascinating few months as we work to build and evolve a platform beyond its existing model.

How will this enhance Magnetic’s product? 

Technology is the heart of our business. Adnamic’s platform technology is a great compliment to our existing infrastructure and will strengthen our product in a way that ultimately benefits our clients. Through this partnership, we will be able to improve campaign performance by making more precise decisions based on deeper insights into our audience search data, and access to complete data sets. In other words, we will become even more effective at reaching more customers who are interested in the message being delivered, which will ultimately drive performance.

What do you hope to achieve by acquiring Adnamic?

We will improve our platform by increasing its ability to operate on larger scale. We will also enrich our analytics platform to consider more data and make richer, more effective calculations about the value of each keyword and audience members for each campaign. Lastly, we will combine our years of experience with greater reach, control and access to data, in order to extend our market-leading position in search retargeting.

How will Adnamic fit into Magnetic’s organization?

The acquisition of Adnamic was a powerful next step for Magnetic’s organization. Our technology integration is already underway and I’m thrilled to welcome Joe Presbrey, founder and CTO of Adnamic, to Magnetic’s engineering team. We share a love of high-performance, distributed computing systems and the ability to extract knowledge from large data sets. And most importantly, we all believe in the same core engineering values: simplicity, efficiency and measurement over argument.

Fast Bulk Data Loading in Mongo

Recently I had to update a mongo collection with the contents of a large file containing JSON objects. This was simple enough in ruby using the native mongo driver, however MongoDB posed an interesting problem.

Due to MongoDB’s global write lock, large updates to a single collection leads to a performance bottle neck. This is unavoidable the first time you run a bulk upload. However, each additional run would require an upsert on every single record, even though only a few records would have changed or need to be added to the collection.

The code below is the solution I came up with:

 

 

 

 

 

 

The main idea is to read each record and generate an MD5 digest, then search your collection for it. If it is not found then perform an upsert with your data and the MD5 digest added. Using this method, only records that are modified will be written, thus minimizing locking issues.

Note: that you should have indexes on the relevent fields in your collection to make the queries performant. In the above example, I have a compound index on first_name & last_name.

The Raspberry Pi

The “Raspberry Pi” is a $35 computer that has been developed over the last few years to be sold into education, and the developing world. The Raspberry Pi is by no means a cut down or stripped computing platform. For $35 dollars you are getting quite a robust feature set:

  • SoC: Broadcom BCM2835 (CPU, GPU, DSP, and SDRAM)
  • CPU: 700 MHz ARM1176JZF-S core (ARM11 family)
  • GPU: Broadcom VideoCore IV, OpenGL ES 2.0, 1080p30 h.264/MPEG-4 AVC high-profile decoder
  • Memory (SDRAM): 256 Megabytes (shared with GPU)
  • USB 2.0 ports: 2 (via integrated USB hub)
  • Video outputs: Composite RCA (PAL & NTSC), HDMI (rev 1.3 & 1.4), raw LCD Panels via DSI 14 HDMI resolutions from 640×350 to 1920×1200 plus various PAL and NTSC standards.
  • Audio outputs:3.5 mm jack, HDMI
  • Onboard storage: SD / MMC / SDIO card slot
  • Onboard network: 10/100 Ethernet (RJ45)
  • Operating Systems: Debian GNU/Linux, Fedora, Arch Linux, RISC OS

So why am I excited?

Powerful.

This is a machine that has real power. At 700mhz with a OpenGL compatible GPU, it is more capable then most machines that were sold as desktops a decade ago. I remember paying $350 dollars (The 3DFX Voodoo 3) just for a graphics card that would match with the GPU that is on this. Here it is playing Quake III Arena in full 1080P mode:

 

Cheap.

There have been attempts to make small computers that are cheap, but nothing with this much power. The Arduino is probably the closest comparison, but the most basic version costs $20 more. It runs at 20MHz with 256K of ram. It doesn’t even have ethernet as standard.

Small.

The Raspberry Pi is incredibly small – at just 3.370 × 2.125 inches it is approximately the size of a credit card. Which means it can be put in tiny things. It is low power and solid state, which means it doesn’t need much juice to run (4 AA batteries will do the trick) and it can take a fair amount of abuse.

I can easily see people turning these into:

I can easily see people turning these into:

  • internet radios
  • cheap NAS appliances
  • robots
  • drones
  • video storage/playback devices
  • custom video arcade machines (this is screaming out for a port of MAME)

Combine all of these and you have the potential for this to inspire a whole new generation of hackers. Much the same way our generation was with 8-bit computers – the Commodore 64, Apple II, and the Timex-Sinclair Spectrum.

I have placed three on order. Can’t wait to get them.

π Day

What is π?

By definition, π is the ratio of the circumference of a circle to its diameter. π is always the same number, no matter which circle you use to compute it.

 

What is π Day?

Pi Day is a holiday commemorating the mathematical constant π (pi). Pi Day is celebrated on March 14 (or 3/14 in month/day date format), since 3, 1 and 4 are the three most significant digits of π in the decimal form. In 2009, the United States House of Representatives supported the designation of Pi Day.

 

Some interesting facts about π

  1. Pi is defined in modern math as the ratio between the circumference and the diameter of a circle in Euclidean Space.
  2. Its irrational, which means that it cannot be written as a fraction of integers, and its decimal form extends past the decimal point indefinitely and without a pattern.
  3. Its transcendental, which means that no polynomial equation exists with rational coefficients such that π is a root of that polynomial. Numbers that can exist as the roots of a polynomial with rational coefficients are called algebraic. Algebraic and transcendental numbers are mutually exclusive subsets encompassing all the real numbers.
  4. Pi has been known since the ancient Greeks and Egyptians. In fact, equations and concepts involving π have been found by archaeologists carved into cave walls well before there was ever a practical application, dating back at least 4,000 years, which reflects the degree of mathematical insight of the human species even in its earliest tribal years. The first numeric approximation dates back to the Babylonians.
  5. Its also called Archimedes Constant. Though less well known here in the English speaking world, the term Archimedes Constant is more well recognized by mathematicians of other languages.
  6. The constant is written and named after the Greek lower case letter π, the 16th letter of the Greek alphabet, whose proper name in English is “pi” and in Greek “piwas”. The letter sounds much like the English letter “P”, which is also the 16th letter of the English alphabet.
  7. The letter was chosen because it is the first letter of the Greek word περίμετρος, which is Greek for Perimeter (as in the perimeter of a circle).
  8. In the early days of its use, π represented the perimeter of a circle (circumference), and not the ratio between the circumference and the diameter (refer to fact 7). It wasn’t until later that pi became defined/assigned as the ratio. Before this time, the ratio was described by its definition. Pi may still be regarded as the circumference of a circle with a diameter of 1.
  9. The letter π wasnt used to refer to the ratio until 1707, when William Jones introduced the letter… and it didnt catch on until 1737 when Leanhard Euler popularized it. Oddly enough, Euler is also responsible for popularizing the use of the letter e for Euler’s number, a constant we have named after him (even though e was discovered by Jacob Bernoulli years earlier); and still no one knows why Euler chose e in the first place, it is unlikely e is an initial of his name.
  10. The letters P and I, which spell the word pi, are the 16th and the 9th letters of the English alphabet. Their sum is 25. Together, 9, 16 and 25 are perfect squares of the values 3, 4 and 5 – which form a Pythagorean Triple.
  11. It might also be interesting to note that the English letters P and I which spell pi have Greek equivalent letters. P and I correspond to the letters π and ι (Pi and Iota) in the Greek alphabet. They are also the 16th and 9th letters of their respective alphabet.
  12. Pi is defined as a ratio on a per-circle basis. Thus, pi could conceivable be different for every circle. It was necessary to separately prove that all circles are similar and thus proportional before we could call pi a constant.
  13. It was Euclid of Alexandria that prove pi was a constant for all circles, and he achieved this using polygons.
  14. Whether or not π has an algebraic relationship to e (Euler’s number) is still an open question.
  15. Whether or not π + e, π − e, πe, π/e, πe, eπ, and other algebraic variations, is a transcendental or algebraic number, is also an open question.
  16. It is not even known if the natural logarithm of pi, ln(π), is even irrational.
  17. Though the expressions of the preceding two facts are not known to be transcendental or not, it is known they are not algebraic for polynomials up to degree 8.
  18. It is actually puzzling that we cannot determine the transcendence of π + e or πe. We know that between π + e and π – e, at least one of the two is transcendental, if not both. Likewise between π + e and πe. In each pair, they cannot both be algebraic else it would imply an untruth: that π or e is also algebraic. We know that π and e are both transcendental, and yet their algebraic combinations escapes all mathematical attempts at resolution.
  19. Pi is not a Liouville number.
  20. The exact area under the function y=e(−x²) is exactly √π. It is only one of three definite integrals of the function which can be evaluated exactly.
  21. Pi is seen in statistics all the time. The value of pi appears in many probability distributions from statistics, including the normal bell curve and the Cauchy distributions.
  22. Pi is seen in probability theory all the time. If you throw a needle 1 unit in length randomly between two parallel lines exactly 1 unit apart, the likelihood of a part of the needle landing on or past one or both parallel lines is exactly 2/π (about 64%). This is called “The Buffon’s Needle problem”.
  23. The process can be reversed, too. If you count how many times the needle crosses a line and divide it by how many attempts were made, for sufficiently large enough attempts, you can compute the approximate probability of success, P. Swapping the formula around 2/P equals π. Here, a random event can be used to estimate a value for pi.
  24. Hundreds if not more different formulas and algorithms have been derived that can compute pi or that can equate to pi without the use of any variables. These are explicit equations and algorithms for pi. Though many of these are computationally unusable or difficult, they are mathematically sound.
  25. Whether or not pi is a normal number is still an open question. A normal number is a decimal number where each digit of the numbering system (0 through 9 in base ten) occur in equal proportion. More accurately, normality is the case where the probabilities associated with the distributions of every possible block of digits occur consistent with the expectations from pure randomness. Pi’s normality is not certain in any integer base numbering system.
  26. It is not even known if each and every digit, 0 through 9, occur infinitely many times within pi. It is clear, though, that at least two distinct digits must.
  27. At the 32nd decimal digit is the first occurrence of a zero. For a number whose digits appear random, this is statistically improbable. Unusual, when considering the fact that the other nine digits are accounted for within the first 13 decimal places, while seven are accounted for in the first 9 places.
  28. As of January 2010, over 2.7-trillion digits of pi have been computed by supercomputers whose sole purpose is to compute pi. Even so, many of these computers only compute from days to weeks at a time before being turned off.
  29. If only one-billion of those digits were printing in 12-point font in a straight line, it would stretch from the middle of New York City to the middle of Kansas.
  30. If the average person were to recite the first one-billion digits without stopping, day and night, it would take a bit over 20 years.
  31. π(x) is called the prime counting function, having nothing to do with the number π, and it returns the quantity of prime numbers less than or equal to x.
  32. The value π does, however, crop up in the distribution of prime numbers.
  33. Computing the digits of pi was one of the first tasks assigned to electronic computers.
  34. Computing the digits of pi have been a fascination and a hobby for mathematicians throughout history.
  35. ∏ is the Greek capital letter for Pi, and is used mathematically as an operator in the same way ∑ (Sigma) is. Whereas the latter is used for sums, the former is used for products.
  36. The capital letter has another use as a function. ∏(x) = Γ(x+1), a horizontal shift of one to the Gamma function, such that the gamma function and the factorial align. ∏(n) = n! for positive integer n.
  37. Furthermore, ∏(−½) = Γ(½) = √π. So you could arguably inform people (although not exactly math-proper to do so) that (−½)!² = π. The last TI-83+ that I played with actually successfully did this using the standard preprogrammed buttons. But the TI-84 and TI-89 did not.
  38. Pi is regarded as one of the most important mathematical constants of all the universe. Right along side the 1 and the 0. Its also right there next to the imaginary unit, i, and Euler’s number, e.
  39. In fact, pi appears in a famous equation called Euler’s Identity: e(i π) + 1 = 0, which ties the five most important constants of math aforementioned into one nice and tidy equation, with no other superfluous values. This identity is regarded by mathematicians as one of the “most beautiful”, and most astonishing. The equality also ties three fundamental operations into one equation: addition, multiplication, and exponentiation.
  40. Pi isnt just used for math with circles. It is important in trigonometry where it forms the basis for a new system of measuring angles. Much of calculus would not be possible without pi. Complex analysis uses pi, and without complex analysis no modern day electronics device would have been invented. Pi is used in a wide variety of other subject matter that has no obvious relationship to a circle.
  41. Lu Chao of China is arguably the man with the most free time. The first 67,890 digits of pi he did memorize, the most in recorded history. His endeavors have been recorded by the Guinness Book of World Records, as of November 19th, 2005; and it took him 24 hours and 4 minutes straight to recite what he had memorized. According to him, it took about one year to memorize. He also claims that he had 100,000 digits memorized but quit on account of making an error on the 67,891st digit. Lu Chao is said to have an excellent memory – he is capable of memorizing a one-hundred digit random number in less than ten minutes.
  42. Math geeks celebrate March 14 as “Pi Day”. March 14 is 3/14 in the American date stamp (month/day). Pi Day is celebrated with pies and pizza pies and any other type of pie, with praise given to the circle, artistic pieces involving pi, and discussions on the history and significance of the value.
  43. The Exploratorium in San Francisco starts their festivities at precisely 1:59 pm PST, a moment on Pi Day that is called “Pi Minute” (3/14 1:59). The Exploratorium was the first to host Pi Day in 1988.
  44. Albert Einstein was born on Pi Day, March 14, 1879 in Ulm Wurttemberg, Germany. Of course, the day was not known as Pi Day at the time.
  45. The Massachusetts Institute of Technology mails out their acceptance letters to be delivered on Pi Day.
  46. For countries that follow the day/month stamp, “Pi Approximation Day” is celebrated July 22 (22/7).
  47. In common years (non-leap years), on April 26th at 4:23:41 AM, the Earth has travelled exactly 2 radians (1/π), or approximately 32% of its orbit around the sun as per the Gregorian calendar system.
  48. In common years, November 10th is the 314th day of the year (3.14).
  49. In common years, December 21st is the 355th day of the year. At 1:13pm we can celebrate a Pi Approximation Minute for the ratio 355/113, an approximation originating in China since 480 AD, accurate to six decimal digits.
  50. On Pi Day, 2009, the US House of Representatives officially recognized March 14th as Pi Day, making the day National Pi Day.
  51. Pi is mentioned in the Bible, but not to precision. They estimated the value at about 3. This can be seen in 1 King 7:24 and again in 2 Chronicles 4:3. But that is not to say the Bible is wrong – there are several theological discussions on this apparent hiccup, including the misinterpretation of the exact design of the object described. Many atheists mock the Bible for this mathematical hiccup as though it were a fallacy of theistic faith. However, depending on the inclusion or exclusion and interpretation of relevant adjacent passages, scholars can arrive at approximations anywhere from 2 to 5 digits of accuracy.
  52. It was the Egyptians who originally invented the 22/7 approximation, accurate to only two decimal digits. An inaccuracy that persists in modern culture to date. Ironically, it was also an Egyptian nearly 700 years later who derived an even less accurate value for which he proudly took credit.
  53. For many centuries mathematicians have attempted to describe pi with rational numbers. This is largely due to the fact that for many centuries the notion of irrational numbers were considered absurd. It was believed all numbers could be described with a fraction, if only we could figure out what the value was! It was Pythagoras who first thought up irrational numbers, proven to exist by one of his underlings, and still mathematicians failed to recognize their existence for many years. It wasnt until decimalized numbers and irrational numbers both became accepted that pi started to be computed to better precision.
  54. The times 3:14 AM and 3:14 PM are both Pi O’Clock AM and Pi O’Clock PM.
  55. An Indiana law proposal (Indiana Pi Bill) was made in 1897 designed to arbitrate the value of pi to 3.2. The Bill actually mentioned three distinct values to be used contextually. The only reason the law was not passed was because a politically active mathematics professor, not even a citizen of the town, happened to be in the court house on his way through the state. This is the only example I could find in United States history in which an attempt was made to legislate physical reality.
  56. According to legend, the aforementioned Indiana Pi Bill has its origins in 1888, when a country medical doctor started to claim he had been divinely taught the exact value of pi.
  57. Pi forms the base of the pinary numbering system, where the integer value 10 in pinary is equal and exact to the value of pi in decimal.
  58. In 1995 a revolutionary new formula was discovered which allows you to compute any specific digit of pi without having to compute any preceding digit – but it only works in hexadecimal (base 16).
  59. We know that the 1-quadrillionth hexadecimal bit of pi is a zero.
  60. Set your calculator to “degree” mode. Type in the number 5 a number of times: “55555″, for example. Reciprocate it with the 1/x button. Then press the “sin” button. The answer will be the number pi shifted to the right a number of decimal places. The number of places pi is shifted is equal to two more than the number of 5′s you originally used. The more digits of five you use, the more pi shifts to the right, but the more accurate pi’s estimate will be!
  61. William Shanks spent over 20 years of his life calculating the digits of pi. In 1873 he finished, having accurately computed pi to 527 digits. He held (and still does hold) the record for most digits computed by hand. His record was only beaten 75 years later by the electronic computer.
  62. The sad thing is that Shanks actually computed 707 digits – it took the electronic computer (a desktop calculator, actually, in an attempt to verify Shanks’ work) to prove in 1944 that the last 180 digits of Shank’s estimate were incorrect. He made a minor arithmetic error on the 528th digit and for nearly 75 years everyone had thought he successfully computed 707 digits.
  63. The first genuine computerized attempt to compute pi from scratch was made in 1949 when John von Neumann used the computer ENIAC to compute the first 2,037 digits of pi, blowing Shanks estimate out of the water by four-fold, and it took the ENIAC only 70 hours to complete.
  64. In 1600, nearly three and a half centuries earlier, Ludolph van Ceulen held the record at only 35 digits, having also spent the majority of his life on this endeavor. He was so proud of his achievement that he actually had the digits inscribed on his tombstone – or so legend has it. He died in 1610 and his grave site has been lost and forgotten.
  65. Due to Ludolph’s endeavors, pi is obscurely known as Ludolph’s Number in the English speaking world. Germans commonly refer to pi as Ludolph’s number.
  66. In 1973 the millionth digit was finally computed – a huge milestone in mathematics, computational science and the science of the electronic computer in general. Fifteen years later in 1989, the billionth digit was finally computed, setting a new record in both computer hardware and software design as well as mathematical algorithms – in particular emphasizing the importance of rapid convergence.
  67. Until the turn of the second millennium AD, only 11 digits of pi had been determined. By 1701 the first 100 digits were known. That is a near ten-fold improvement in under a thousand years when for more than three thousand years beforehand only ten digits were discovered.
  68. Nine centuries before the birth of Christ, the value of pi was only known to two decimal places. It took until 2 centuries after Christ for Claudius Ptolemaeus (Ptolemy) to evaluate one additional digit.
  69. In 1168 AD, a Judaic scholar by the name of Maimonides explicitly stated that pi could only be known approximately. This is the first historically documented assertion that pi was irrational. Though true, it could not have been proven nor known with certainty at the time.
  70. It wasn’t until 1768 that Johann Lambert proved pi’s irrationality.
  71. In 1794, Legendre proved that π² was also irrational.
  72. Ferdinand Lindemann, in 1882, proved that π was transcendental.
  73. Archimedes, who died 212 BC, actually pinned pi into a range of 223/71 and 22/7. He achieved this by using polygons instead of circles.
  74. Archimedes is also the man who discovered that the area of a circle was pi times the square of the radius (A=πr²). This was achieved using triangles.
  75. The digits of pi have passed all known tests for randomness. Tests include the distribution of the numbers, the probabilities of particular sequence combinations, and a wide variety of others.
  76. In the first 6 billion digits of pi, each digit is statistically expected to occur 600 million times. And yet, on average, the even digits occur about 15 thousand times less than expected while the odd digits occur about 15 thousand times more than expected. Unusual as that might be, no digit occurs outside the expected deviation. These deviations are disregarded as statistically insignificant.
  77. In the first 100,000 digits of pi, the digit 1 occurs most frequently and the digit 9 occurs least frequently.
  78. There is a book titled “Joy of Pi” which is about pi. There is also a movie “Pi”.
  79. There is a relationship between π, the Fibonacci numbers and the Golden Ratio – quite astonishing.
  80. The Feynman Point is the 763rd decimal position of pi, where for six consecutive digits the number 9 appears.
  81. The three-dimensional equivalent definition of pi – the ratio between the surface area of a sphere and the area of the largest cross-sectional circle – has a value of exactly 4.
  82. Similarly, in spherical geometry (non-Euclidean space), pi is exactly equal to 2 even though it is still defined as the ratio between circumference and diameter of a circle.
  83. Pi is originally defined for a circle in Euclidean space. We use the approximation for pi, 3.14159, to whatever number of desired digits we wish, for all practical applications. And yet Einstein proved that space is not Euclidean after all, it is curved. Pi is incorrectly valued for the space we apply it in.
  84. The word “pi” is sung in “I am the very model of a modern major general.” The digits of pi are sung (incorrectly) in a song titled “Pi”, which was a recent production by a popular band. Other songs to pi can be found on youtube.com, made by teachers and mathematicians intending to make math more fun.
  85. The Great Pyramids at Giza are modeled after pi (among other things). The vertical height of the pyramid has the same relationship to the perimeter of its base as the radius of a circle has to the circumference.
  86. Starting at the letter H, write down all the letters of the English alphabet in alphabetical order. Capitalized block-letter form. When you get to Z, cycle back around to A and finish off the alphabet at G. Erase all the letters that have left-right symmetry: A, H, I, M, O, T, U, V, W, X, and Y. The remaining letters are now grouped in sets sized: 3, 1, 4, 1, 6.
  87. The letter P is the 16th letter and I is the 9th letter of the English alphabet. Their product is 144, a perfect square, as is 9 and 16. The first 144 digits of pi add up to the Devils number: 666. The sequence “666″ doesnt occur until the 2,440th digit.
  88. Including only digits past the decimal point, the sequence “314159″ doesnt occur until the 176,451st position. The first five digits past the decimal “14159″ dont occur a second time until the 6,955th position.
  89. The 359th decimal digit of pi, which is the 360th digit in total if you count the 3 in the ones place, begins the same three digits: 360. Since pi is intimately connected with the circle, it is curious to note that there are also 360 degrees about a circle.
  90. Pi was the secret code that Alfred Hitchcock chose to use in both his Torn Curtain and in his The Net (which was a movie staring Sandra Bullock)
  91. A Givenchy brand men’s cologne is named Pi and sports the symbol π. The cologne is marketed as highlighting the sex appeal of intelligent men.
  92. The function π(x) isn’t just the previously mentioned prime counting function. It is also the form we use for a different function entirely. π(x) = 1/Π(x), the reciprocal of Π(x), where Π(x) is the horizontal shift in the gamma function: Π(x) = Γ(x+1).
  93. The upside down capital Pi, ∐, is an operator called the Coproduct of a set.
  94. The word “pi” or a sequence of its digits have been sung in a number of university football cheers and cadences.
  95. Pi, of course being a Greek letter, is found among the names of many sorority and fraternity housing chapters.
  96. As a generalized continued fraction, pi can indeed be expressed with a simple, straightforward pattern.
  97. In the famous OJ Simpson trial, a defence attorney, Robert Blasier, attempted to discredit an FBI agent by proving his ignorance of basic science. He initiated a debate with the FBI agent about the “true” value of pi.
  98. Pi is an important theme in Carl Sagan’s novel Contact.
  99. In a Star Trek episode titled “Wolf in the Fold”, Spock foils the evil computer by commanding it to compute the last digit of pi.
  100. Pi is a running source of humor in many math oriented jokes within math culture. Probably the most overused and amateur of which are the ones comparing pi to the pastry pie.
  101. Computers and their components are stress tested by having them compute the digits of pi. Pi computing is a famous technique for assessing a computers efficiency and speed.
  102. Poets and theologians have scorned the mathematical endeavor to determine the digits of pi as an attempt to rationalize God and detract from natural beauty.
  103. Other poets and theologians praise the endeavor as a more empirically rooted pursuit of divine truth and the realization of natural beauty.
  104. “The Pi-Search Page” is a web page which attempts to find any user-inputed sequence of digits embedded within pi.
  105. “My Slice of Pi” is a web page in which you can purchase a digit of pi for the same US dollar amount as the numeric value of the digit you are buying. Numbers are purchased in order. Zeros are free. And each digit bought may be turned into a link and an advertisement.
  106. “Pi10k” is a web page which attempts to convert the digits of pi into a musical sequence whereby you may “hear” for audible patterns.
  107. The circumference of a circle encompassing the known universe can be computed within an error the distance of a hydrogen atom with only 39 digits of pi. You will need 43 digits to get accuracy to the width of a proton. It makes you wonder why we bother with trillions.
  108. Most real-world engineering situations require no more than 7 digits of pi for accuracy, and most require far less. NASA’s space missions, including orbitals and satellites, rely on only 7 digits of precision.
  109. The exception is the Hubble Space telescope which utilized an engineering record 10 whole decimal digits (in 32-bit binary). The Hubble’s mathematical preciseness allows for astronomical readings of one ten-thousandth (0.0001) of an arc-second accuracy (reduced to seven thousandth (0.007) of an arc-second due solely to other physical limitations such as material composition). Hubble’s focus wont budge the width of a human hair seen from a mile away.
  110. According to legend, Archimedes of Syracuse was working on pi computations when Roman soldiers, in 212 BC under the command of General Marcus Claudius Marcellus, took the city. Archimedes didnt notice the invasion and was rude to a soldier who interrupted him. Archimedes supposedly said “Noli turbare circulos meos!” (“Dont mess up my circles!”). The soldier decapitated Archimedes even though the army was under orders not to harm him – General Marcellus respected learned men.
  111. Some mystics believe that there is a divine message hidden in the digits of pi, which is also the theme in the previously mentioned Carl Sagan’s book Contact, although Sagan gave it a more scientific and less theological feel.
  112. Some gamblers believe that by understanding pi well enough, they can beat the odds at computerized games which generate pseudo-random numbers from the digits of pi.
  113. A crop circle in Britain, known as the Barbury Castle Crop Circle, formed on June 1, 2008, became famous as it is one of the most complex crop circle ever documented. It is a symbolic representation of the first ten digits of pi (nine past the decimal), the last of which is inaccurate due to rounding. It took the help of astrophysicists to figure out that the symbol represented pi. The crop circle is often compared to the crop circles of 1996 in Avebury Trusloe, Wiltshire, which were fractal patterns of the Mandelbrot Set and the Julia Set.
  114. The half-derivative of the square root function is a constant: √(π)/2
  115. If it took you 16 whole minutes to utter the number 3. Then 8 whole minutes to utter the number 1. Then 4 whole minutes to utter the number 4. Then 2 whole minutes to utter the number 1. So on, so forth, each time uttering the next successive digit of pi in exactly half the time of the previous. Then it would take you a little over a half an hour, 32 minutes to be precise, to say all of pi in its entirety! After the 32nd minute you would be finished.

Install Octopress on Lion

After upgrading to Lion (MacOS X 10.7.3) I encountered a few problems getting Octopress running.

The problems I had encountered are due to Lion switching to LLVM-GCC from the default GCC 4.2 compiler in the previous versions of MacOS X. This causes failures when compiling Ruby 1.9.2.

After much head scratching and researching on Google & Github I came to the following steps.

You will need to install the GCC tool chain – you can download it form here. Download GCC-10.7-v2.pkg and install it

All Ruby versions should be built with GCC 4.2 instead of LLVM-GCC. On Lion, /usr/bin/gcc is linked to /usr/bin/llvm-gcc-4.2. So we need to set the default compiler to GCC-4.2 by typing before running rvm install:

  • export CC=/usr/bin/gcc-4.2

Open the gemfile in your Octopress directory and replace

  • gem ‘rb-fsevent’

with

  • gem ‘rb-fsevent’, :git => ‘https://github.com/thibaudgg/rb-fsevent.git’, :tag => “v0.9.0.pre4″

and save it.

Finally, type

  • bundle install

You should now have a working version of Octopress.

 

 

 

 

 

 

MongoDB Analytics for Online Advertising at Magnetic


On Wednesday, August 10, Mark Weiss, Sr. Director of Engineering at Magnetic participated in a webinar hosted by
10gen. In the webinar, Mark examined how Magnetic uses MongoDB for their online advertising platform.

Mark discussed how the company has simplified its architecture by taking advantage of MongoDB’s rich combination of features. The webinar covered the following key topics:

- How Magnetic takes advantage of MongoDB’s flexible data model

- Leveraging atomic counters for real-time analytics on impression counts and frequency caps

- Using map/reduce to offer a new tier of near-real-time reporting

See the complete presentation here >>


Practice and Theory: Search Retargeting Company Magnetic Shares its Approach to Hiring Great Engineers


Like any engineering profession, software engineering is driven by both theory and practice. The Magnetic interview process reflects this by evaluating candidates’ judgment and experience in building working systems and also their foundation in fundamental software engineering and computer science principles.

The first step in the interview process is to get a clear understanding of the level of experience within engineering.

Asking candidates to discuss their experience helps us evaluate what they have learned in past professional situations, how they approach solving problems under typical professional constraints, how they work on teams, and how much they taken away from one project and apply it to the next.

Technical screen questions are also valuable, but for a different reason. By focusing on general software engineering issues and computer science fundamentals like architecture, software design, algorithms and data structures, we can look at candidates as a group. Where discussion of experience discriminates candidates as individuals, tech questions level candidates relative to their peers.

In addition, theoretical fundamentals provide a common framework for analyzing problems and understanding solutions. We screen on fundamentals because we want to observe candidates applying them to solve problems. We want a sense of the tools a candidate chooses when presented with a new problem to solve, and how much mastery they have working with those tools.

For this reason, we favor starting the process with phone screen questions, which are deceptively simple – that is, simple enough to explain quickly, but subtle enough that better candidates identify and address subtleties as the discussion proceeds. The best candidates point out issues that have never come up before, or highlight flaws in the answers presented to them. One of the most interesting and rewarding aspects of interviewing is how much you learn from the interviewees!

Candidates who do well in the phone screen move on to visit Magnetic’s Times Square offices for an on-site interview. This provides us an additional opportunity to evaluate the candidate as an individual and in turn, allows the candidate to meet the team, spend time in the office and get a feel for working here. Our engineering team growing quickly, so each team member’s enthusiasm and commitment crucially contributes to the team’s success.

On site, we also present candidates with a classic “go to the whiteboard” technical question. This exercise combines theory and practice because the candidate is working through a general question, while literally “thinking on their feet.”

Finally, we ask candidates to bring in code they have worked on and do a code review with us. This has proven to be a very powerful evaluation tool. Excellent and conscientious engineers should be able to discuss tradeoffs, strengths, and weaknesses of software they have created. They should be able to present code in the context of the goals it was trying to achieve and the constraints under which it was written. The code review is as close as we can come to getting a feel for the candidate as a day-to-day developer.

After all that, if we’re excited about you joining Magnetic and you’re excited by the team you’ve met, our engineering challenges, our focused technical strategy, and the outsized business opportunities we’re pursuing, then we’ve made a match.

Welcome to the team!


Search Retargeting & Magnetic Part 1: The Two Jobs of a Great Engineering Team

It isn’t hard to explain at a high level what Magnetic does. In the simplest formulation, we use search data to target online advertising across the Internet. That is the mission of our business.

To succeed in our mission we need to solve many engineering and business problems. First, we must provide our customers with access to enough of the right data for them to effectively target to meet their campaign goals. To do this, we give marketers a scalable, powerful and yet simplified solution to create, manage and review the success of their campaigns. Additionally, with billions of targeting actions per month being executed across Magnetic’s platform, we need to analyze our data in order to identify ways to better serve our customers.

So, while the high-level description of our business might seem straightforward, the engineering and product problems aren’t so simple. Above all, there are two rules we use to distill the responsibilities of Magnetic Engineering into a core team mission statement: keep our products running, and continuously improve our products.

DevOps

Keeping things running has traditionally been in the realm of operations, the engineering sub team responsible for keeping systems available and performing well. At Magnetic we follow the principles of DevOps, which requires developers to be responsible for supporting the software they create, and in general, integrates operations with development to a great degree. As we grow, there will be dedicated operations engineers, but application developers will never stop supporting operations.

All engineers at Magnetic are responsible for supporting applications released to customers. Developers test releases for performance and performance under load. Additionally, they use automated tools to release their software to customers and build monitoring and alerting into their applications

Product Development

Product development serves as the engineering team function focused on improving products and developing new ones. Magnetic uses the Scrum methodology for product development. Agile approaches such as Scrum posit that the engineering team’s primary responsibility is to regularly release software changes that increase the value of the company’s products.

Agile is a great fit for Magnetic. Most importantly, it keeps the team focused on what matters most — our customers. By releasing often, we quickly deliver new features, smoothly integrate with new business partners, and proactively make infrastructure improvements that keep our systems working well.

Agile also allows Magnetic Engineering to be nimble, enabling us to identify goals and priorities, and then focus on execution. Because we break work up into manageable chunks, we can plan ahead without committing to months of work, leveraging that nimble ability to shift focus to newly identify higher-priority work. On the other hand, we can break larger projects into a series of smaller, more predictable steps, and get feedback from customers as we complete them.

A great example of this is the Magnetic SEM tool currently in alpha release. A full-featured SEM product requires significant effort, but core features useful to alpha customers can be built more quickly. Currently, our engineers are working directly with customers to identify the next set of innovative features to add to the tool. This highlights two more core Agile principles we leverage at Magnetic — interact directly with customers to understand their needs, and build working “thin slices” of applications as early as possible.

As the head of engineering, nothing is more satisfying than seeing our engineers own the development of a tool and drive the expansion of our business. I’m doing my job only to the extent that there are no barriers to the team achieving its two missions of keeping our systems running and continuing to add features to our product that make it more valuable to more customers.

In my next post, I’ll discuss the connection between system architecture and business success at Magnetic.

Cheers,

Mark Weiss

Senior Director, Engineering at Magnetic