We’ve all heard the phrase, “correlation does not imply causation”, but no one ever talks about what causation really is. Fundamental physics has an answer for us, and it might surprise you: It all comes back to correlation. It’s a rainy day in the city, and everyone has their umbrellas…

Complex systems are ubiquitous in nature, and physicists have found great success using thermodynamics to study these system. Machine learning can be very complex, so can we use thermodynamics to understand it? — As a theoretical physicist turned data scientist, people often ask me how relevant my academic training was. While it is true that my ability to calculate particle interactions and understand the structures of our Universe have no direct relevance in my daily work, the physics intuitions that I learned are…

Describing reality as a simulation vastly understates the complexities of our world. Here’s why the simulation hypothesis is a lousy one — With the advent of artificial intelligence, and ever-increasing computational power, it seems like there’s no limit to what computers can do. From hyper-realistic 3D rendering to deepfake images and videos (e.g., thispersondoesnotexist), the possibility of fully simulating reality seems to be right around the corner.

How can we use BigQuery to handle tables with many columns? Here’s a how using scripting and table metadata. — BigQuery (BQ) is a Petabyte-ready proprietary Google data warehouse product. Pretty much every day in my work, I use BQ to process billions of rows of data. But sometimes, tricky situations arise. For example, what if my table has a large number of rows and columns? …

Quantum physics doesn’t have to be weird. Here, we explore a modern and intuitive interpretation of how quantum systems really work. — The word “quantum” is often associated with complicated equations and unintuitive physical phenomena. Yet, our world is unapologetically quantum; quantum physics governs all that exists in the Universe. …

Learn how to supercharge your aggregation queries using Materialized View — BigQuery (BQ) is Google’s proprietary data warehouse product, advertised to be ready at the PetaByte (PB) scale. However, it’s not immediately obvious how to scale to PB. In fact, looking at the cost structure of BQ ($5/TB), running PB-scale analytics seems prohibitively expensive, as querying 1PB of data will quickly…

Many people think the arrow of time can be explained by the increase of entropy; but that’s inadequate. We’ll explore why. — “An inch of time is an inch of gold”: The transience of time has been known since time immemorial. Unlike trekking across an open plain or diving into the ocean’s depths, one can neither explore around nor remain stationary in time. …

The story of discovery from the perspective of a physicist in training. — I can still vividly recall the events of July 4th, 2012. The alarm clock woke me up around 3 a.m. I was slightly groggy, but I quickly collected myself, remembering the significance of the moment. I was still a graduate student at Princeton at the time. There was a celebration…

Unlike slot machines, BigQuery lets you generate random numbers for FREE! Here’s how to leverage this to perform free computations — BigQuery provides a convenient and cheap serverless framework to run data analytics and algorithms at scale (you can sign-up for a free account here). However, its SQL frontend might seem like a rather stringent constraint. No worries, there is a hack out of this: custom User-Defined Functions (UDFs) in Javascript.