Context Matters! Lessons Learned From YouTube Personalized Recommendations (And Why This Matters For Twitter)

“But what I do have are a very particular set of skills; skills I have acquired over a very long career.”

Was tweeting the other day about how Twitter’s “Who to follow” recommendations lacked context and speculated the inclusion of more information might increase conversion. Here’s what the web version looks like today:

who to follow

While the suggested accounts might be personalized by Twitter’s understanding of my interests, and in shel’s case it provides a bit of data (someone I follow follows him), on the whole it’s pretty basic. I feel no excitement, no engagement and have largely gone blind to the unit. Twitter is full of smart people so I don’t doubt whatever they have is optimized but perhaps to a local maxima (or at least the effort<>reward of this growth feature). What would be more interesting to me? An interesting exchange between someone I follow and someone I don’t with a recommendation to follow the latter. A single account with their last – or most popular – tweet displayed alongside so I’m enticed to follow SoftTechVC or Bitcoin. Some variation in the display that causes me to check it out each time.

Besides gut instinct why do I think Twitter might want to explore these ideas? Well, it would be that aforementioned ‘very particular set of skills.’ During my years at YouTube I had the opportunity to observe many of our experiments in providing context. One that feels particularly relevant was our own launch of recommended videos based on your viewing history. While it performed fairly well at launch there was one tweak made early on which increased clickthrough and perceived relevance: we told you why we were recommending a specific video. Although the UX has since changed to remove this additional information, here’s an example from several years back:

youtube_recommendations

You can see the “because you watched: <title>” displayed (although this isn’t a great screenshot since it cuts off most of titles, but you get the general idea). The exact numbers are hazy but I recall when we launched this additional information, two things happened:

  • Clickthrough on the unit went up ~20% over previous results (and page CTR increased as well so it was additional clicks, not pure cannibalization – although there was some cannibalization)
  • When we recommended a video someone didn’t like, the user blamed themselves and not us!

The first benefit is self-explanatory but the second one was unexpected, amusing and deserves a bit of elaboration. Let’s say you come to your YouTube homepage one day and we’re recommending you watch a Justin Bieber video. “No way” you might say, “I’m NOT a Bieber fan. YouTube’s personalized recommendations suxxxx.” When we showed you that same recommendation but told you we were recommending it because you had watched Miley Cyrus “Party in the USA,” all of a sudden the user was all “um yeah, I guess I did. Heh heh.” Now of course our goal was to provide the right video, something they’d enjoy and click on, but it was interesting to see the change in perception this context provided (an impact we deemed as positive).

So I’m generally a fan of experimenting with additional context when it provides a look inside your algorithmic black box. I think it makes your product seem more human to its users and feels like your algorithms are working on behalf of the person and their interests rather than just treating them as a row in a user database.

Would be very interested to hear about other people’s experiences that had similar or different outcomes with regards to context.

The Ethicist: VC Edition

Sunday brings the NYTimes and the Sunday Times brings The Ethicist, a column currently helmed by culture writer Chuck Klosterman. While the questions debated are widely varying in subject matter, I don’t recall any addressing venture capital. Maybe the Secret app is our version of The Ethicist but rather than publishing anonymously, let me post here: What assumptions should entrepreneurs make with regards to VCs and confidential information?

It’s a question which on the surface seems simple but given the amount of information investors receive and the different pathways which bring it to them, I’m not sure the expectations are universal or absolute. Here’s how I’ve operated at Homebrew. Curious whether folks behave differently or have feedback for me.

If it’s information that has come from a founder as part of a deal pitch, it’s ALWAYS held in confidence. And if I believe we might be conflicted, I tell the founder at the earliest moment possible, before any other information is shared (interestingly, the two times we were potentially conflicted, the founders (a) thanked us for being so upfront and (b) still wanted to share more info with me so we could determine whether there was a conflict). My commitment to this principle holds true regardless of whether I’ve approached the company proactively or it’s a cold email from the founder. Just because they provided me confidential info prior to my request, I still believe every founder should assume my inbox is a lockbox. If you put something there as part of pitching your company, I’m going to respect that trust, even if the information would assist one of my portfolio companies. I can think of one asterisk to my claim here: if I’m made aware of a company via a pitch, I might share any publicly available info (such as a placeholder URL or Twitter account) with someone but not with any additional context drawn from the confidential info. Here’s an example:

Company ABC.com sends me a deck unprompted describing their future vision for 3D printing. I had not heard of ABC.com prior but upon going to their website see that’s it’s a placeholder with tagline “inventing the future of 3D printing.” In this case, if there’s a company in our portfolio or an entrepreneur I know which might be interested in what ABC.com is doing (collaboratively or competitively), I may pass along the URL and just say “keep an eye on these guys.” That’s it. If the recipient wants to know more, I’ll decline but if they want to introduce themselves to ABC.com founder, I’d tell them to send me an intro email and I’d pass along to the founder, without necessarily revealing who it is, etc. To me that seems fair.

What if I receive confidential information indirectly, for example, another VC passing along a deck or a friend telling me about something happening at a company, or to a person, etc. To me this is a more grey area. I’m sure there are some people who would suggest that if the subject themselves didn’t want you to have the information you should treat it as confidential by extension. I understand that opinion and think it’s admirable but can’t honestly say I fully live by it. At the same time, I do accept some responsibility that the fact this information has reached me is (a) not at the request of the subject and (b) using it inappropriately might cause problems for the subject. Since my goal is to be karma-positive (or at least kindness-positive), I want to balance self-interest with impact.

Great, great but what does that actually mean for how I use this information (if at all)? Usually in one of two ways. If it’s a case of wanting to reach out to the subject directly based on what I’ve learned, I’ll ask the person who gave me the info whether I can do so and to what degree I can suggest that I’m informed. There’s the “I know so let’s talk.” There’s the “hey thinking of you and wondering whether it’s a good time to talk.” And sometimes I just need to sit on it out of respect for the situation.

The other way I’ll use it is in a watered down version, likely not forwarding on the confidential deck or specific details but perhaps reaching out to the impacted Homebrew company and say something like, “hey, I hear BigCo ABC is building a product similar to yours. Have you heard anything like this? If not, might be worth digging a bit.” Would I ever pass along the full info verbatim? I don’t want to say a definite “no” but it’s not my default behavior.

Overall in all of these situations I think it’s essential to optimize for two goals: (A) that people can approach me proactively and in an unguarded fashion because I’m going to protect their interests and (B) even if I’ve invested in your company, I’m going to balance your self-interest with trying to keep the tech community a positive, safe space.

What do you think? Are there other questions involving VCs that you want to ask about? I’ll answer as best I can with the caveat that I’m early in my career so haven’t encountered the range of scenarios that a 20 year investor may have.

Bet On People Prematurely

Approaching Homebrew’s one-year anniversary which means getting some materials prepared to update our investors. Taking stock of the last twelve months emphasized two things for me. First, how excited we are about the founders we’ve backed and how easy that makes working hard on their behalf. Second, how fortunate I am that along the way in my career, certain people have helped out. For example, David Hornik who I *think* was the first VC I encountered post-graduate school in 2001 when he was poking around my employer Second Life. David’s proclivity for being “nice” is well-documented in Adam Grant’s book Give & Take and in my case it translated to spending time answering questions, inviting me to events, and generally giving me the Donnie Brasco “he’s a friend of mine” treatment. He bet on me prematurely.

Betting on people prematurely factors a lot into my current business too. We back plenty of first time founders. Their passion isn’t going to wait so we get on board, hang on and try to clear the path ahead of them so they can move as quickly as possible.

Betting on people prematurely can also be a great hiring strategy. Find the superstar who’s just a bit less experienced than your job req suggests and give them the chance to step up. Especially from larger companies where promotion criteria is too firmly specified, good folks can find themselves locked into junior roles for distressingly long periods of time. Bring them on board, elevate them but also provide the support they need. You might need to spend more time with them ongoing, not micro-managing but helping structure and think through their decisions. Find them a functional mentor, either within your organization or outside. Invest a little in training if there’s a specific area they want to build confidence. Often I don’t want to bet on someone who has done the job before, but someone with the headroom and capacity to do this job and beyond.

Gil Penchina’s Army: How AngelList’s Largest Syndicate Plans to Lead A Rounds, Hire An Associate & Keep VCs Honest

Gil Penchina doesn’t appear in TechCrunch or Valleywag often. He tweets infrequently. Maybe you haven’t heard of him until now. But he’s raised the largest syndicate on AngelList, turning himself effectively into a one-man fund for, if not the masses, at least the 270 people who have already committed nearly $3.4m for each investment he wants to make. 

Gil cut his tech teeth as an early eBay exec and I met him first several years ago when he was CEO of Wikia. Having followed AngelList Syndicates pretty closely (I believe Homebrew was the first institutional fund to invest alongside one), Gil’s multimillion dollar path and his future plans, strike me as pretty exciting and undercovered by the tech press. 

Q: Although your $3.3m AngelList syndicate has surprised some people, you’ve been angel investing for quite a while, right?

I did my first angel investment in 1998, so 16 years and 60+ investments.  I’m not focused on making a log of noise but honestly it wasn’t too surprising to me when you consider:

  • my [angel] portfolio is up 5x cash-on-cash

  • most of my peers from the early 2000 days of angel investing are now VC’s (Reid Hoffman, Peter Thiel, Josh Kopelman, Jeff Clavier, Mike Maples, Dave Whorton, etc).  I was one of the few not to raise a fund

  • I had 9,000 angel list followers many of who had tried to co-invest with me previously

Q: When AL launched syndicates, what was your immediate reaction? Had you been thinking about raising a fund via other channels?

I had thought about raising a fund several times and always decided not to.  The main reason was that having a fund requires spending a lot of time with lawyers and accountants. Process people make me crazy since entrepreneurship is all about challenging the established order of things.

Q: Secrets of your success? Why do you think you’ve raised the largest syndicate thus far?

It’s a combination of many entrepreneurs I’ve helped over the years who owe me one, investors who I’ve traded deals with in the past and an enormous amount of trust from people I don’t know.  It’s actually a bit intimidating when you realize the burden that places on you.

Two of my three syndicate deals have already had an up round from well known VC’s.  The Andreessen and Founders Fund investment in AltSchool is public.  The other one I can’t talk about yet, but we’re very excited about it. So clearly picking good deals has helped both build trust with my co-investors in the syndicate and attract more commitments.

Q: Can you talk a bit about the composition of your syndicate – people you know or strangers? Small commitments or large commitments? Do you let anyone join?

Join today at: https://angel.co/penchina#syndicate

Anyone can join. I have $1,000 backers and big money backers. Dentists from the Midwest, tech CEOs, Facebook engineers, VC’s and private equity partners.   What’s great is they all want to help too.  With advice, referrals, customer intros and more.

Q: $3.3m+ is a stunning amount – fully deployed it would allow you lead an A Round. Do you think of yourself as being competitive with VC firms? Have you elbowed any VCs out of deals?

I’ve already been working on 2 A rounds. I recently lost the chance to lead one, but because my “better offer” let the entrepreneur improve the price and terms from the VC…  he called it a GATNA (Gil Alternative to A Negotiated Agreement).  I also think we may be an ideal bridge round partner for companies that need a little more money before an A since I don’t need board seats and am generally stupidly entrepreneur friendly.

Our syndicate has already lead and priced a large seed round twice. Those deals would have been an A a few years ago ($1mm+). That said I fully expect to lead an A in the next year.

Q: Has being more public changed your dealflow or the nature of conversations entrepreneurs, coinvestors are having with you?

I see entrepreneurs reaching out to have me join their round because a hot deal on AngelList raises their company’s profile to VC’s and employees.  That let’s me get small investments in some of the best entrepreneurs

AngelList is also a very high profile service, so being a top angel has many benefits

  • more entrepreneurs reach out now

  • I continue to attract new funds at a crazy rate.  I expect in a year we could legitimately do B rounds too

  • as a credibility badge, saying 250+ people think I’m a great angel is a fantastic signal to entrepreneurs.  As are the comments entrepreneurs have left on my angellist profile page at https://angel.co/penchina

Co-investors have occasionally been a little offput.  I’ve attended a couple board meetings where a bridge round was discussed and the VC’s work together as a club to set the price rather than bidding against each other.  In both cases I’ve stated that I want to bid against them to give the entrepreneur a fair market price.  In one case a VC ranted at me “this is an insider round!!”  I responded with a smile “yes, and I think I’m an insider”.   I ended up bidding the deal up to almost twice what the “insiders” had generously offered the CEO.  So if you know someone frustrated with the “inside round” offer they get from their VC’s, tell them to contact me on LinkedIn or Facebook and we can “fix” a low ball offer

Q: Advice for someone who is thinking about joining a syndicate?

Think about the fact that VC’s need 10 deals to have 1 winner.  In my experience angels need 25-30 deals to get that same winner… So don’t bet your whole savings on one deal :-). I have counseled several angels to cut back on their commitment to me so they can do a smaller amount in more deals and spread their risk.

Q: Any predictions for what Syndicates will look like 12, 24 months from now?

I am attempting to expand the definition of the syndicates in several ways today:

  • A rounds

  • buying common from founders

  • bridge rounds

Assuming the current crazy growth continues I fully expect to start bidding on B rounds next year. I’ve already hired an intern and am interested in talking to people who would want to be an Associate for my virtual VC firm.

gil-penchina-headshot

 

Funding Probabilities: Are You One of The Best 10? 100? 1000? Companies Raising This Year?

“The partner is likely writing just two or three checks this year. How are you going to convince them you’re one of those companies?” This was the way a friend described their mentality in pitching VCs on A, B or C round financing. What he meant was, the average venture capitalist at those stages will only be funding a few companies each year but seeing many pitches. In that sense it’s not just about proving that you’re a great opportunity, but also that you’re worth the opportunity cost of using one of those “slots.”

I’m thinking more deeply about the A Round fundraising process as the set of companies we’ve backed start to mature. At least three will be out in market this year for their next financings, and in other cases, pre-emptive interest is starting to bubble. The potential I want our companies to communicate isn’t just their own traction, roadmap and strength of vision but that they’re one of the best opportunities the VC is going to see this year. How do you signal that and what are some strategies?

I don’t think it should be a large part of your initial deck but do recommend a few minutes and single slide (or whiteboard sketch!) to educate the VC about the dynamics of the market you’re in and why the timing for disruption makes sense. A founder’s unique insights are always captivating – at least to me.

For some it’s also about matching the type of risk inherent in your business to the partner or firm who (a) is comfortable with that risk and/or (b) believes they can help solve it. For example, your billion dollar outcome might have some regulatory speedbumps along the way. For some firms that’s a very scary issue and maybe even a deal killer. But others feel like they’ve seen those navigated before and perhaps even can assist you in doing so. By looking at their current portfolio and talking with some of their founders you can map which partners like/dislike specific issues (note, in some cases the reverse might be true -> continuing the example above, if their portfolio is already full of regulatory-challenged companies and they don’t want to bear any more of the same risk).

Finally, and the most direct, is just naming this during your pitch process. “Look, I know you see a lot of deals and the sheer law of numbers say that you’re not going to fund most of them, even ones you like. We believe we’re one of the 10 best opportunities you’re going to see this year for reasons A, B and C. Do you agree? What else do you need to figure out in order to agree with us?” Of course some investors might be put on their heels with your directness but I think that’s fine – you’re not being inappropriate, you’re trying to find out how their process works and what questions they have. The investor who can’t articulate what they’re trying to learn from you is an investor who is unlikely to ever commit!

What are some ways you’ve been able to signal to investors that you’re one of the 10 or 100 best companies raising this year?

As an aside, here’s how Homebrew’s process works:

Stage 1 – you meet with Satya or myself. We’re taking the meeting because based on what we know about you, we think we might be a good match for your company. The goal of phase 1 is to see if we can get from *think* to “this IS interesting, pending some more discussion around points A, B, C”

Stage 2 – with those specific points named, the other partner joins the conversation so you can get a sense for both of us. We try to work through the open questions – with you, with data or references you provide and on our own. Our goal is also to make this part of the process feel like working together, not pitching one another, because we’re also trying to give you a feel of what it’s like to work with Homebrew.

Stage 3 – even if we’re still working through some of the points, when we reach the point of conviction we’ll provide you guidance on the way we were thinking about participating in the financing (terms). That helps you react in a concrete manner and also know where you stand.

We’re still tuning all of this for efficiency and speed.

Never Been Easier to Become a VC. Never Been Harder to Stay One.

Recently attended the Upfront Summit, an LA-area tech event put on by Upfront Ventures, which also serves as their annual meeting for LPs to get updated on the fund’s performance. As we approach Homebrew’s one year anniversary we’ll do a smaller scale meeting with our investors in San Francisco. Part of our presentation will be portfolio financials, which, because we’re relatively new, aren’t exceptionally volatile (LP speak: most of our investments are still carried at original value since no additional fundraising has occurred).

During this past year I’ve seen many new firms get created and seem to always have a few friends who are in the process of considering VC. Sometimes it feels like it’s never been easier to become a VC but never harder to stay one. There’s a influx of capital to back new funds:

  • crowding funding via AngelList syndicates
  • high net worth individuals
  • family offices
  • strategic investors (including corporates)
  • other venture/private equity shops
  • traditional institutional LPs such as fund of funds, endowments, foundations, sovereign wealth funds (our investors)

The result has been many new managers, many new funds. Not all will be in business five to 10 years from now.

Venture funds are like startups that play out in slow motion because we raise funding every 30-36 months, compared to 12-18 month cycles for an operating company. First fund is like a seed round – it’s based on pitch and reputation. Second fund is like an A round – momentum and beginning of traction. Third fund is the B round – metrics, results. That’s when new funds tend to hit a wall.

We’ve had a strong first year but it would be hubris to suggest I can tell you what our returns look like just yet. I know the founders we’re seeing are strong enough that a great fund could be constructed on top of the opportunities presented, so that feels great.

But this is a hard job. Sure, not coal miner hard, not startup founder hard. And it’s still an amazing opportunity to invest other people’s money without putting all of your own at risk (we are investors in Homebrew personally so there’s skin in the game). But seed venture, at least the way we play it, isn’t a bunch of smiles, $50k checks in hot deals and parties. It’s being a partner of conviction to an entrepreneur and working with them for several years. Succeeding with them but getting paid last.

It’s not a lifestyle job. And I love it.

Day Before Opower IPO, SVP Product Alex Kinnier Reflects on Journey

Energy efficiency company Opower is ready for its public debut this Friday. In advance of the IPO I reconnected with their SVP Product & User Experience Alex Kinnier, whom I know from our days together at Google. I couldn’t ask Alex specifically about the IPO or future plans but wanted to get some more info on a company that gets less press than your typical social app but is poised for an amazing run.

Q: How do you describe Opower to the average person?

We help utility customers take control of their energy use. How? We make software incorporating behavioral and data science. This SaaS offering is now used by nearly 100 utility companies worldwide to give their customers personalized analysis of how they’re using energy and advises them on how to use less.  In 2013, we helped utilities and their customers save over 1,900 gigawatt hours of energy.

Q: Companies like Nest sell direct to consumers whereas Opower tends to work directly with utilities. How do you compare these strategies? 

Opower’s direct utility partnerships provide us with massive reach enabling us to achieve outsized environmental and business impact. Right now our SaaS platform is reaching 32 million households and businesses. Because we house all of the utility’s meter data, we’re able to analyze usage and behavioral data on those customers plus tens of millions more.

We use all this data to power the behavioral insights—like how your energy usage compares with your neighbors—that nudge consumer behavior.

In contrast, when going straight to consumer, like Nest, you simply can’t build a dataset that grows with the same speed and volume as ours is. Additionally, Nest is more focused on revolutionizing household devices then they are energy consumption. And, in my opinion they are doing a great job with the household devices.

Q: What was your path from Google to Opower and are there other Xooglers at the company?

Just like you Hunter, I love building cool things with great people. At Google I got to help build Google Toolbar and Google’s large advertiser ad server. I then went into venture capital. As a Partner at Khosla Ventures I got to help teams like Climate Corp, Skybox Imaging, Solum, and Pellion build world changing businesses. I was then a Partner at NEA where I was helping another set of great teams, like Avos and then Opower. At Opower I clicked with the founders, the culture, and the mission in such a profound way that when they asked me to join—I saw it as a once in a lifetime opportunity that I couldn’t pass up.

I am extremely fortunate to work with 12 Xooglers at Opower. We all came because of the environmental mission and huge business opportunity. Several that you know are Wayne Lin, Senior Director of Product Management, Shawn Thompson, Director of Professional Services, and Barry Stern, VP Finance.

Q: Are you seeing more technical product managers, engineers and designers get interested in energy-related companies? I mean it’s very different than the latest mobile social app.

Absolutely, I mean which would make you feel more satisfied at the end of day? Writing code to create a new jellybean on Candycrush, or writing code that will help reduce global warming pollution by a billion pounds of CO2 this year? We’re seeing a big spike in technical candidates who want to put their talents toward making the world a better place. Last Earth Day, we calculated that every Opower employee’s annual impact is equivalent to taking all their Facebook friends’ homes (average of about 500 of friends) off the grid for a year.  Our measurable double-bottom line impact is a powerful asset when it comes to recruiting top talent.

Q: Is US leading the way on this type of connect home technology or are there other countries who have also embraced the software approach to energy efficiency?

It’s a global phenomenon. Utilities around the world are pursuing innovative approaches to achieving energy efficiency goals and improving customer engagement. In the last 3 years, we’ve expanded to partnering with utilities in eight countries, including the UK, Japan, and Australia. Utilities in each of these countries, especially in competitive markets, understand that software-fueled programs can help keep their customers happy, manage the grid optimally, and meet regulatory goals. We’ve seen that energy consumers’ desire for personalized efficiency tools and advice from their utility is something that transcends borders. That’s why our cloud software solution and its behavioral science underpinnings are gaining traction in all corners of the globe.