Making Meaningful Metrics Matter More

We’ve been talking a lot about dashboard metrics and making them meaningful. If you manage to accomplish this, you can get people to do some amazing things. It’s a way to provide clear direction, show progress, create impact in the right areas and have something to work toward besides raw financial success.

The downside is that a metric that no one can clear affect, no matter what impact it actually has, will either get ignored or acting on in an unintended way.

For example, at work we have a recycling indicator that measures recycling as a percentage of waste.

It’s likely easier to affect this indicator for us all to bring our recycling to work than to reduce our waste or recycle more of our waste. So, there is an odd incentive to control the indicator with unintended methods.

Food waste is likely our largest source of waste, by far. And, while I don’t know what the cafes are doing with composting, it’s not included in the indicator at all, and yet would probably have a much bigger impact to support a composting program (we also have gardens on site now, they would certainly benefit as well). Also, the output of the cafes is counted, but that’s essentially a different business, so we have influence there, but not control over their processes.

Lastly, there’s a serious problem of my own impact. I produce very little waste personally. And, I’m not alone. It’s very likely that this is a “long tail” problem, where a small minority of the population produces the majority of the waste. The awareness of our waste stream is great, but it’s not clear what most of us can or should do to improve the situation and meet the goal.

No indicator is perfect, and so it’s important to really think through the implications of setting a goal based on a calculation. And, sometimes it’s easier to measure something as a proxy indicator to measure something else. However, it’s important to recognize the difference between the proxy measure and the real measure.

A prime example of this is seen in measuring the country’s economic health. This has been a leading story for quite some time, and we use the Dow Jones Industrial Average as a central measure for the overall economy. Admittedly, it’s not a horrible proxy measure, but it’s definitely not really measuring the overall economy.

DJIA vs. S&P 500 - close correlation despite very little overlap 

The reason it’s a good proxy is that the stock market tends to self-correlate, meaning that, on average, all stocks perform about the same. (Note the “on average” in that statement; a single stock doesn’t have this behavior). So, to get an idea of how the entire market is doing, you only need to average a handful of stocks together to get the general trend. This is exactly what DJIA does, averaging (very crudely) the top 30 companies on the market.

The reason it’s a bad proxy is that the stock market doesn’t necessarily represent the economic health of the nation though. DJIA just hit an all-time high, and yet I don’t think anyone would say that we’re seeing an economic boom right now. Again, there is likely a long tail when considering the population as a whole. The stock market is probably a great indicator of how the upper class is doing (and actually, is probably a very direct measure of this). But, this is a measure of how wealth is doing, which is not evening distributed. So, the increasing divide in wealth would explain why the stock market can sore and the majority of us don’t feel it. I digress; it’s another case of a bad indicator.

I’ve given two examples of bad indicators (and I could keep going). In both examples, a contributing problem is a long tail, which essentially concentrates what’s being measured with a minority of stakeholders. This isn’t a bad thing in itself. After all, our intension is to recycle more of our waste at work, so the minority of “worst-offenders” should get more attention. However, if the point of the indicator is to change behavior across a large population then the contribution of each individual needs to be considered.

To measure and change behavior, a much better recycling indicator would measure each person’s contribution. Maybe something like this:

This is not a perfect measure. But, it’s one that measures individual contribution much better, and gives each of us a very clear direction on how we can change the indicator.

So, making indicators can get incredibly involved. The impact you want to have with them can be subtle, perverse, powerful, or ignored. The focus on what that impact should be is very important when making these. Sometimes the impact of making a change is the most important piece, sometimes awareness and behavior is more important. I hope I’ve shown you that there’s much more to this that just coming up with a fancy formula.

Women In The Tech World

At the May intensive, we got to see a presentation from Team Tinkr for their idea to promote more women to enter STEM (Science, Technology, Engineering, and Math) fields. I wanted to write a bit of a follow-up to that, because it’s something I would love to see change.

They are attempting to encourage young women to stay interested in the tech fields through middle and high school in hopes that this will get them over the hump and stick with it into college. I think this is definitely an important issue to address, but it’s only one aspect of a bigger system.

Keeping the interest of young women in technical fields is one piece, and correcting the larger issue of why they lose interest in the first place is also important. There certainly seems to be some bias built into our culture to create the environment where women don’t want to stick with these fields.

In contrast, this isn’t a problem everywhere in the world. There are a couple countries that are near parity. After visiting Israel, it was quite an eye-opening experience to see no discernible gap in gender representation. So, there are areas where this isn’t a problem (or at least, much less so) which strengthens the case that the main drivers are culturally based as opposed to some more “natural” explanation (I can’t even hypothesize on what that might be as a plausible theory anyway).

I see at least two other major factors that contribute to the male bias.

Keeping Boys In

I don’t necessarily think there is some force actively trying to keep women out of STEM other than latent cultural stereotypes. This may be a prevailing force, but I think it’s better explained by actively keeping boys interested early on.

A great example of this is Lego, which has been marketed to boys almost exclusively for many years. After all, if Lego’s target market isn’t white boys named Zac[k], I don’t know who they’re after. And, it might be a little bit of a leap, but I’d be shocked if there wasn’t a correlation between playing with Lego and learning some core technical principles (I’m not alone on this thought: education.com).

Anita Sarkeesian of Feminist Frequency lays this out in her video on the subject far better than I do.

Reinforcing Culture, Current Tech Culture

It’s natural that any system with a bias will probably reinforce that bias in some way. The current culture in the tech industry is definitely male-centric, and could very well drive young women out of the industry altogether. This doesn’t seem to be on purpose, but a male-dominated field is going to have a misrepresentation through the ranks.

So, the superstars and role models in the field are going to push the bias onward, even when they don’t personally want to. I personally look up to Richard Dawkins and Neil deGrasse Tyson as prominent leaders and communicators in science. These are just two examples and there are countless more, but the prevalence of male STEM role models prevails, which isn’t doing any favors for trying to inspire young women. I wouldn’t expect this bias to just be gender-based. If, for example, all prominent STEM leaders were short or had hair, I’d probably feel like there isn’t a future for me.

Why Would This Be Better?

STEM jobs are some of the most useful, stable, high-paying jobs in the world (in my opinion, but c’mon, it’s true). So, there is that (hopefully) obvious benefit and rationale for wanting to change the system. But, there’s also the benefit of various viewpoints. I’m a strong believer in diverse viewpoints promoting better innovation. So, all those gender-based cultural differences can be used in a better capacity.

In my experience, the women I’ve worked with have shown a surprisingly different and creative thinking than most of the men (including myself). Most companies have figured this out and are promoting more equal representation, so hopefully this effect can help encourage more women to commit to a STEM career.

Disclaimer

I know this can be a sensitive subject. If I’ve said something offensive or insensitive, it was done completely out of ignorance. I’m constantly learning, so please bear with me.

The Principle of Practical

Most disagreements can be characterized as a fight between the principle versus the practical. Consider this short list:

Principle	Practical
…the right of the people to keep and bear arms shall not be infringed. 2nd Amendment	The U.S. has far more gun-related killings than any other developed country. Source:   washingtonpost.com
Irresponsible behavior was rewarded, failure was bailed out, and the innocent -- people who have nothing whatsoever to do with the banks -- suffered Read more: upi.com	Congress wrangles over how best to avoid financial Armageddon Source: economist.com
Corporations should be protected by the Bill of Rights.	Corporations having the same rights as people leads to many unintended consequences like too much political influence.
We shouldn’t put any more greenhouse gases into the atmosphere than the planet can naturally remove.	If we suddenly curtailed all air pollution tomorrow, our economy would be ruined and lots of people would suffer and die.
Cutting down trees is bad.	Much of the US is actually over-populated with trees and thinning them would result in better fire control and more water. Source: whitmanpioneer.com

The conflict in many of these situations arises from attempting to apply a principle to a real situation and having to deal with the reality and unintended consequences of a decision. During the financial crisis in 2008, our lawmakers were scrambling just to contain the problem, so they went with a very imperfect solution. One reason many absolutely hate TARP is because it violates many of our own principles. Most notably is the principle of fairness, which I believe most of us hold.

Ron Paul may be one of the most principled politicians in US politics, and vocally so. He makes the case for extremely principled decisions with this quote:

“I argue the case that if 1 percent of the people need food stamps, you give up 100 percent of the principle”

Source: esquire.com

He applies this logic to many areas, leading the House of Representatives in no-votes. Source: thepoliticalguide.com.

Principle isn’t a bad thing to have at all. It would be impossible to create any framework for government (or business for that matter) without a set of guiding principles. However, we get some pretty absurd outcomes when we follow principles blindly. The gun regulation debate illustrates this. In principle, the government shouldn’t be in the business of regulating guns. But, with such huge problems with gun-related crime, some latitude seems necessary. This is where the practical part comes in.

Congressional approval is abysmal, at around 13% (source: gallup.com). One major reason is the lack of compromise (source: cnn.com). This is where acting solely on principle breaks down.

Ron Paul’s characterization of giving up 100% of the principle isn’t right. Principles can be guidelines instead of absolute rules. Allowing for no compromise will ultimately lead to stagnation as congress can’t agree on anything and there is no path to coming to agreement.

I’m not arguing that we should abandon principle, or allow for the degradation of our rights. But, I am arguing that if we do something that is a compromise on our principles, we understand why that is and strive to improve. Congress is an easy target, but disagreements with principle vs. practical happen all the time. Unfortunately, all too often this becomes an impasse due to the rigidity of the principle.

I think we all need to recognize that our principles are very important, but there is always the reality to consider as well. Doing something outside of your principles isn’t necessarily bad if you understand why that happens and treat those principles as goals to strive for.

Kitchen Remodel

I recently got my kitchen remodeled in an effort to get my condo ready to sell. I wanted to take this as an opportunity to talk about my experience. This whole experience is so topical to class that it’s shocking.

My goal was to get the kitchen in “good enough” shape to sell since it was the room that needed the biggest improvement. I suspect this will aid in getting a better sale price, we’ll see soon enough. I initially budgeted $5000 for the remodel, which included refinishing most of the cabinets, the floor and putting it a countertop that was a little higher quality than the existing one.

I hired a guy to do all this, and while he didn’t provide me with a concrete estimate (mistake #1), he was confident that the work and budget were reasonable. We also agreed that the job would take 2-3 weeks, again given the assurance that this may not be exact, but it is reasonable. The deal was that I pay for materials and labor at an hourly rate. He’s not an experienced contractor, but I don’t mind working with a guy trying to get started and help out a local.

Ultimately, the job took about 6 weeks and over $9000. Ouch! While that’s certainly not ideal, it’s interesting looking back what I should’ve done differently. I haven’t worked with a ton of contractors directly, so I might have unrealistic expectations on what they will agree to. However, my experience with the contractor that remodeled my bathroom was far better in this respect, and I think I pinpointed three interdependent reasons why.

Risk Ownership

One issue that didn’t occur to me when agreeing to the flat hourly rate was that I owned all the risk. I don’t expect the contractor to assume all the risk either, but any delay to the work, any problems, and changes to the plan (even a non-existent one) fall on me. In retrospect, this is wasn’t a wise move. One reason for that is that it’s extremely unlikely to reap the rewards of this risk. Are there really any serious expectations that a job like this will go significantly under budget? Yeah, right.

In class, as we keep going through valuation of business and risk analysis, generally everyone is looking to at least balance risk and reward, if not find opportunities with high reward for low risk (a feat that if I could do reliably would negate the need for school, and I’d currently be having a beer with Warren Buffet).

So, given a chance to redo this remodel, I’d probably insist on some way to mitigate the risk ownership. On the previous remodel, I’m sure I paid for labor somewhat directly (it isn’t clear from his invoice), but this was also never discussed. We agreed on a budget and work to be done and it was done. So, either everything stayed on schedule and I didn’t notice that I paid direct labor or he was just charging me for the work and assuming all the risk on his own. Unfortunately, I just don’t have enough data to know what a reasonable labor payment scheme that shares the risk appropriately would be.

Estimation

Next, we never agreed on a clear estimation of the work required. He had concerns about how much time the cabinet refinishing would take. Aside from that, I never got a clear estimation on how much and how long this would actually take. This ties to the risk ownership in that I agreed to the job with no estimation and all the risk.

We’ve been working a lot on valuation, which definitely requires some reasonable expectations. I’m curious to see how this goes in practice. It seems incredibly difficult, and everyone says that you end up throwing out the plan on day one anyway. As Mike Tyson has said, “everyone has a plan until they get punched in the mouth.”

We ended up never even trying to estimate the job in any formal sense. In retrospect, even with some serious unknowns, I wish I had written down some estimates, it would have naturally created triggers when the estimates were off. Which leads to…

Measurement

Throughout the weeks of work, I’d ask, “how we doin?” and get a response, “alright, some minor hiccups but we’re making progress.” Sounds good, now that that’s under control, time to get back to accounting homework, right? Well, lesson learned. Had I actually been tracking the labor hours along the way, I wouldn’t have been shocked by it at the end. Near the end of the project, he gives me all his worked hours, and I about spewed tea all over the place. When I was at around the budget of the project, it was roughly half done! The huge shock of this was not a welcome surprise.

In the end, I was a horrible manager. I was in charge, I had the ability to collect the necessary data, and I decided to work on homework instead. Lesson learned, homework is never the answer. With tongue removed from cheek, I do have to admit this was a valuable learning experience on setting expectations, measuring progress and dealing with some difficult conversations.

Charts

More dimensions is more awesome, right?

Worst.Chart.Ever.

Cellphone As A Service

A cell phone. Also, not competitive.

There are over 300 million cell phones in the US. Given that most of those phones will only have a working life of under 3 years, that's a lot of e-waste. Even worse, because of the construction of most cell phones, recycling the materials is really difficult. Consumers (and thus manufacturers) want cell phones to be as small and compact as possible. So, the integration of metal, plastic, glass, circuit board, etc. is very specialized and not easy to unmanufacture.

This isn't an easy problem to solve. As cell phones get better, faster and smaller, consumers will continue to toss a lightly used obsolete brick for the next new shiny smartphone. It's not reasonable to expect consumers to settle for a substandard cell, so we need a better solution to deal with this ongoing problem. It'd be nice if there was a solid business model in it too.

To start, someone needs to take ownership of the full lifecycle of these devices. What I'm talking about is cradle to cradle (C2C) instead of cradle to grave. Given that there isn't barely any effort in handling the end of this lifecycle, cradle to grave is the going paradigm.

Few companies exhibit C2C thinking better than Herman Miller. They design office furniture with the end of the useful life in mind from the beginning. Commonly, furniture can be refurbished or dismantled and recycled. While it's probably really difficult to make a consumer electronic with this in mind, putting the ownership of the end of the lifecycle on the manufacturer gives them the incentive to work on this problem.

One way to do that would be to restructure cell phone contracts. Instead of selling cell phones with a contract to subsidize it, why not simply rent them to consumers? A couple things work a lot better with this scheme. One, if a consumer is willing to pay the premium price, they get the top-end phone, no matter when it shows up. No more waiting for your two-year contract to expire. Then, tiered pricing can allow for those that don't want or need the top end to get the leftovers that do. This also adds value in avoiding the dreaded 2-year contract. If you want to quit, turn in your phone and move on.

Why would a cell phone provider want to do this? Well, instead of making money constantly churning out more sales (a growth model), they make money by providing good service. Ultimately, they can remix and reuse the devices whenever a new one comes out. So, they are able to provide a better service at a premium price and have a lower capital cost (due to reusing the waterfalled devices).

It gets better. Now that the carrier has ownership of the devices, they now have incentive to push for better designs with respect to C2C. They could take the reuse idea a step further. Commonly, the most expensive part in a phone is the screen. And, with a simple glass refinish, could be made as good as new from an obsolete phone. The biggest thing keeping this from happening today is that the phones aren't designed with this in mind. With a realignment of incentives, it wouldn't take long for carriers and manufactures to figure this issue out.

And, in the end would reduce e-waste considerably, because the carriers and manufacturers who do it best make the most money. Wouldn't it be nice if the big bad tech companies start fighting for the environment and get rewarded for doing it right?

Avoiding Camels

We’ve been talking a lot about marketing, market research market intelligence. One major component for performing this research is focus groups; or more broadly, gaining insight into your target market. However, relying on your market to define your product can be very dangerous. How do you avoid letting your market research turn your offering into a camel?

This is genuinely an open question. It’s very difficult to avoid. I’ve been involved in some massive camels.

The Simpsons captured this phenomenon perfectly. In the episode “The Homer”, Homer is tasked with designing the perfect car because, after all, he represents the middle of the target market. The result is an utter disaster.

There is a huge challenge in asking your market what they want (in my opinion), because most of the feedback is solution oriented instead of problem oriented. Ideo (http://www.ideo.com/us/), the design firm behind many creative products and including just about everything that Apple produces, has perfected this process.

I can’t completely break down their design process, but there’s one key takeaway I’ve learned- look at the extremes of your target market. It seems logical to find people right at the heart of your target market and focus on pleasing them. Then, expand from there. Ultimately, this is probably what you’ll end up doing, but look to the people that are on the fringes, the early adopters, experts, late adopters, etc. After trying this in a very limited way at work, I’m starting to see the genius of it.

At work, I produce data analysis reports, charts, tables and generally anything you can do in Excel. The life of a rock star isn’t what everyone expects, what can I say? In this work, I get random people all over my organization asking for my services. But, I almost always get a solution dictated to me. And this, ladies and gents, produces camels that are basically useful to no one.

There are guidelines for developing these sorts of things, this is an example of ignoring all of them. (To be fair, you are looking at gibberish data and I clipped off some important parts for security reasons). The point is, this chart is a serious camel. It has probably been redesigned a dozen times, with a ton of user feedback. Market research is not the issue here; this is exactly what they’ve asked for. And yet, it’s nowhere close to what they want.

Since then, I’ve started focusing on the extremes. Specifically, two user types: the experts who know the underlying data and novice users who don’t necessarily even know what the chart is measuring. By focusing on these 2 groups, some very good things started happening.

One, the development time dropped dramatically. This is likely due to needing feedback from fewer people. But also, it’s easier to get deeper feedback from this reduced size of the feedback group.

Two, the users providing feedback were much better about describing the problem rather than the solution. As a result, I think it’s the middle that suffers from the “solution” thinking the most. In retrospect, that makes sense. The experts know exactly what they’re looking for, what’s possible, and can directly describe the problem (in many cases they are incredibly knowledgeable about the problem itself). And, the novices don’t understand the solution at all, they only know the problem. Pay dirt.

It’s those pesky “middle” users that know enough to be dangerous. But, it seems, please the extremes and you can drag the middle along for free.

This is the result of working with the extremes. Just aesthetically, this chart is much easier to read and understand (again, a lot of clipping and gibberish data undermine my point here). And, it addresses the major use cases much more effectively.