REDstack is Now Open Source! We are officially open sourcing REDstack, our sandbox tool for Big Data development at Target. What is REDstack? REDstack is a tool for provisioning kerberized clusters on OpenStack. We created it with four goals in mind: Provide a secured environment, with the ability t...

K8Guard is Officially Open Source I am happy to announce that Target has open sourced K8Guard. I have been part of designing and developing it for the past few months, and I’m going to share a little more about it. What is K8Guard? K8Guard is an auditing system for Kubernetes clusters. It monitors d...

Here at Target, we run our own private OpenStack cloud and have never been able to accurately measure the performance of our hardware. This lack of measurement prevents the evaluation of performance improvements of new hardware or alternative technologies running as drivers inside OpenStack. It also...

One of the strongest benefits of launching an application into the cloud is the pure on-demand scalability that it provides. I’ve had the privilege of working with the ELK stack (Elasticsearch, Logstash, Kibana) for purposes of log aggregation for the past two years. When we started at that time, we...

It is 7:30 AM on a Monday morning in late October. I am waiting in line at Cafe Donuts to bring my team breakfast for our mandated ‘no work for one hour’. We just wrapped up a strenuous week of implementing a major upgrade to our Elastic logging cluster. Many digital teams are relying on this upgrad...

Hadoop upgrades over the last few years meant long outages where the Big Data platform team would shutdown the cluster, perform the upgrade, start services and then complete validation before notifying users it was ok to resume activity. This approach is a typical pattern for major upgrades even out...

Background Just after the middle of last year, Target expanded beyond its on-prem infrastructure and began deploying portions of target.com to the cloud. The deployment platform was homegrown (codename Houston), and was backed wholly by our public cloud provider. While in some aspects that platform ...

Target’s open source big data platform contains a vast array of clustered technologies or ecosystems working together. Troubleshooting an issue within a single ecosystem is a difficult task let alone an issue that spans several ecosystems. It is impractical for a single human to individually investi...

Win the cloud with Winnaker! I am happy to announce that we, at Target, decided to open source a tool called Winnaker. This tool will allow the user to audit Spinnaker from an end user point of view. But first what is Spinnaker? The first time I heard the word Spinnaker, my reaction was, “wait, what...

At Target we aim to make shopping more fun and relevant for our guests through extensive use of data – and believe me, we have lots of data! Tens of millions of guests and hundreds of thousands of items lead to billions of transactions and interactions. We regularly employ a number of different mach...

On my first encounter with it, around early 2010’s, I was mystified. It sounded like witchcraft and I imagined the practitioners to be a coven of witches and wizards, all holding Ph.D.s in the dark art of “Data Science” and being respectfully addressed as “Data Scientists”. It was believed they woul...

When you first think about scaling an on-premise Hadoop cluster your mind jumps to the process and the teams involved in building the servers, the time needed for configuring them and then the stability required while getting them into the cluster. Here at Target that process used to be measured in ...

An enterprise as large as Target generates a lot of data and on my Big Data platform team we want to make it as easy as possible for our users to get it into Hadoop in real-time. I want to discuss how we are starting to approach this problem, what we’ve done so far, and what is still to come. Our re...

At Target we’re always looking for ways to move forward in becoming the best omni-channel retailer that we can be. This journey demands that we enable change in most every part of our technology organization. Our culture, delivery model, technology selections, working arrangements and org structures...

In 2014, I worked on a project to build out a system to provide new business capabilities for our Marketing and Digital spaces. This initiative was not unlike other initiatives we undertake at Target, but it provides a great example of the importance of us investing in an effort to modernize the way...

One important part of a retailer is their supply chain network. In order to sell a product in our stores, or online at Target.com, we need to have a well run and maintained network to move all of those products from one place to another. It may not sound like a complex problem to ship products from ...

Setting the stage Celebrity guests in the house Jez Humble opens the day with a bang.The best in the business. Ducy opens with a goat. To the shock of many, the rest of the talk was goat free. Deming's obligatory appearance Record attendance Heather Mickman and Ross Clanton opening and closing the d...

Recently we launched Project Argus, a 30-day “monitoring challenge” to improve visibility of key performance indicators (KPIs) across our technology stack prior to our peak retail season (in Greek Mythology, Argus is a 100-eyed giant). This effort was structured as a mix between a FlashBuild and agi...

At a couple of recent conferences around DevOps (MSP DevOps Days and DevOps Enterprise Summit), Heather Mickman and Ross Clanton from Target explained some high level concepts and actions we’re taking to break our mold of working on complex problems. When presented with an opportunity to work differ...

Tuesday, October 14th 2015, Target affirmed its commitment to technology and leadership in the retail space by joining the World Wide Web Consortium (W3C). As a company that cares about delivering great experiences for customers, we realize those experiences are now steeped in technology. One way Ta...