{"id":61,"date":"2012-06-17T00:09:00","date_gmt":"2012-06-17T00:09:00","guid":{"rendered":"http:\/\/www.guythomas.org.uk\/blog\/?p=61"},"modified":"2023-11-17T15:43:17","modified_gmt":"2023-11-17T15:43:17","slug":"investment-and-optimal-error-rates-moonshots-and-mundanes","status":"publish","type":"post","link":"https:\/\/www.guythomas.org.uk\/blog\/investment-and-optimal-error-rates-moonshots-and-mundanes\/","title":{"rendered":"Investment and optimal error rates: moonshots and mundanes"},"content":{"rendered":"\n

The main point of this note is to suggest that an investor should have a higher tolerance for false positive classifications when selecting shares with right-skew and\/or fat-tailed returns<\/strong> (potential \u201cmoonshots\u201d). <\/p>\n\n\n\n

Terminological preamble<\/strong><\/p>\n\n\n\n

Statistical discussions of hypothesis testing commonly refer to “false positive” (Type I) and “false negative” (Type II)\u00a0errors.<\/strong>\u00a0The term \u201cerror\u201d is value-neutral in statistical testing, but may be unhelpful in a portfolio selection context, because it might be misread as insinuating some analytical mistake on the part of the investor.\u00a0 To avoid this blameworthy connotation, I will use the (slightly) more neutral term \u201cdisappointment.\u201d \u00a0\u00a0<\/p>\n\n\n\n

I seek to avoid\u00a0ex-ante<\/em>\u00a0errors\u00a0<\/strong>of analysis, whilst recognising that there is an optimal rate of\u00a0ex-post<\/em>\u00a0disappointments<\/strong>. (I can’t think of a succint word pair for the distinction I am stressing here: on the one hand blameworthy\u00a0ex-ante<\/em>\u00a0errors<\/strong>, and on the other hand blameless\u00a0ex-post<\/em>\u00a0disappointments.<\/strong>\u00a0 This seems an interesting linguistic lacuna.)<\/p>\n\n\n\n

Portfolio selection as a classification problem<\/strong><\/p>\n\n\n\n

Portfolio selection can then be viewed as a classification problem, with \u201cpositives\u201d being shares which are added to (or retained in) your portolio, and \u201cnegatives\u201d being shares which are rejected (or sold). Normally in classification problems you seek to minimise the error rate, defined as a weighted sum of false positives and false negatives.  The two weights \u2013 one for false positives, and one for false negatives \u2013 are set according to the cost of each type of disappointment.<\/p>\n\n\n\n

To give an example from medicine: if a disease is potentially fatal but has a reliable and safe treatment, then false negatives are costly, and false positives are benign.  Hence we should tolerate a higher rate of false positives<\/strong> (eg pap smear testing for cervical cancer in young women).   On the other hand, if a condition is often inconsequential and the treatment tends to be worse than the disease, converse payoffs would apply and we should tolerate a higher rate of false negatives<\/strong> (eg PSA antigen testing for prostate cancer in elderly men). <\/p>\n\n\n\n

In portfolio selection, false positives and false negatives cannot sensibly be identified with just two payoffs.\u00a0 Instead each type of disappointment generates a return distribution. The nature of that distribution depends on what type of share it is: a moonshot<\/strong> or a mundane.<\/strong> The graphs below show the return distribution for an individual stock drawn at random from each class.\u00a0<\/p>\n\n\n\n

Moonshots<\/strong>\u00a0are shares with the potential for very high growth: shares which might \u201cgo to the moon\u201d. Say Apple in 1997, or ASOS in 2003, or QXL in 2005 (of course examples are easy to identify ex-post!). \u00a0Potential moonshots are few in number. Most moonshots fizzle out (call these \u201cduds\u201d).\u00a0 Actual moonshots are very rare (call these \u201chits\u201d).<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

Mundanes<\/strong> are (unleveraged versions of) housebuilders, manufacturers, engineers \u2013 reliable businesses, but not plausible moonshots.  Mundanes are plentiful, and easy to recognise.  The nature of their operations makes long-term high growth unlikely.  Compared to potential moonshots, mundanes produce outcomes which are less extreme, and more evenly distributed between \u201chits\u201d and \u201cduds\u201d.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n\n

Is portfolio selection more like pap smear testing (false negatives are more costly), or PSA antigen testing (false positives are more costly)?\u00a0 The answer depends on the type of share:\u00a0false negatives have much higher (opportunity) costs for moonshots than mundanes.\u00a0\u00a0<\/strong>For moonshots, a false negative means we miss one of the very few Apple-like shares which could make a big difference to our portfolio return. For mundanes, false negatives are not much of a problem, because a single missed mundane won\u2019t make much difference to our portfolio return, and there are always plenty more largely interchangeable mundanes we could include.\u00a0 \u00a0<\/p>\n\n\n\n

This implies we should apply a\u00a0heavier penalty to false negatives<\/strong>\u00a0(and therefore necessarily also\u00a0accept more false positives<\/strong>) when classifying\u00a0moonshots<\/strong>\u00a0than when classifying mundanes.<\/p>\n\n\n\n

Toy example: portfolio selection from moonshots<\/strong><\/p>\n\n\n\n

Suppose that the entire class of moonshots comprises 100 shares, with these payoffs: 90 out of 100 go bust, and the other ten rise to 3x their cost.\u00a0<\/p>\n\n\n\n

Assume we select 10 shares for our portfolio, and make equal investments in each of them. <\/p>\n\n\n\n

If we select at random, the expected portfolio return is a loss of 70% (0.1 x3 x1 + 0.1 x0 x9).<\/p>\n\n\n\n

But if we can manage to select four true moonshots, the expected portfolio return becomes a positive 20% (0.1 x 3 x 4 + 0.1 x 0 x 6).  With five true moonshots, it\u2019s 50%. With 6,7,8,9, 10, it\u2019s then 80%, 110%, 140%, 170%, 200%.<\/p>\n\n\n\n

So in this (extreme) example, we have can have a false positive rate as high as 60% when selecting mooonshots, and yet still generate a positive portfolio return.<\/p>\n\n\n\n

Would it be better to tighten our criteria, and so reduce the false positive rate below 60%?\u00a0 That would reduce false positives (duds). But it would also increase false negatives (hits). Because hits are so rare, the combination of reducing false positives and increasing false negatives may produce a portfolio with a lower fraction of hits, and hence a lower expected return.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Toy example: portfolio selection from mundanes<\/strong><\/p>\n\n\n\n

Suppose the entire class of mundanes comprises 500 shares, with these payoffs: 250 give a 20% loss, and 250 give a 30% gain. (Note that there are 5 times as many mundanes as potential moonshots.)<\/p>\n\n\n\n

As before, we select 10 shares for our portfolio, and make equal investments in each of them. <\/p>\n\n\n\n

If we select at random, the expected portfolio return is 5%.<\/p>\n\n\n\n

If we select mundanes with a 60% false positive rate \u2013 the same disappointment-tolerant strategy which produced a positive 20% return from the potential moonshots class above \u2013 then our expected return is nil (0.1 x 0.8 x 6 + 0.1 x 1.3  x 4 = 1.0).  In this case, a 60% false positive rate produces a lower result than chance; the disappointment-tolerant selection strategy which worked for moonshots doesn\u2019t work for mundanes.<\/p>\n\n\n\n

To achieve a positive return from mundanes, we need to penalise false positives more heavily. Say we tighten our selection criteria to reduce the false positive rate to 40%. Then our expected return is 0.1 x 0.8 x 4 + 0.1 x 1.3 x 6 = 10%.  With the false positive rates of 30%, 20%, 10%, and 0%, the expected returns are 15%, 20%, 25%, and 30%.<\/p>\n\n\n\n

By tightening our selection criteria, we also probably increase the false negative rate – that is we reject some hits. But we don\u2019t care much, because no single mundane makes a big difference to the portfolio, and there are always hundreds more mundane hits to find. <\/p>\n\n\n\n

Other observations<\/strong><\/p>\n\n\n\n

In advocating raised tolerance for false positives when selecting potential moonshots, I am not<\/strong> saying that we should set out to make careless judgments.  We should strive to avoid ex-ante<\/em> errors of analysis; but we also need to accept that even diligent judgement may lead to a high rate of ex-post<\/em> disappointments, and we need to be comfortable with this pattern of outcomes.   <\/p>\n\n\n\n

A problem with advocating higher tolerance for false positives for selecting potential moonshots is that shares are not labelled as belonging to one or other of these categories.  The categorisation is itself a matter of judgment. I have no solution to this.<\/p>\n\n\n\n

How do we increase the false positive rate for moonshots?\u00a0 The most obvious way is just to be (a little) more credulous when assessing potential moonshots, and conversely for mundanes.\u00a0 To formalise this, one can use looser requirements for current financial metrics when assessing moonshots.\u00a0<\/p>\n\n\n\n

Another way might be to apply an inclusive checklist for potential moonshots, and a disqualifying checklist for mundanes.<\/strong> <\/p>\n\n\n\n

By inclusive checklist<\/strong> I mean that the presence of certain positive features (say a management team with exceptional previous start-up success) guarantees inclusion<\/strong> in the portfolio largely irrespective of other any concerns. By disqualifying checklist<\/strong> I mean that the presence of certain negative features (say Debt > 3 x EBITDA, or large share sales by insiders) guarantees exclusion <\/strong>from the portfolio, irrespective of any other merits of the company.<\/p>\n\n\n\n

One extant manifestation of my suggested strategy \u201cfatter-tailed and\/or right-skew returns => be more tolerant of false positives\u201d is the tech start-up sector.\u00a0 For angel investors in tech start-ups, most investments are duds, but they hope to more than make up for this with a few big hits.\u00a0 Peter Thiel (of Paypal \/ Facebook fame)\u00a0suggests<\/a>\u00a0that to a first approximation, an angel investors achieve a positive portfolio return only<\/strong> if their single best investment ends up being worth\u00a0more than all the others combined.\u00a0<\/em>\u00a0<\/p>\n\n\n\n

Update (21 October 2012): Paul Graham makes much the same point: angel investors in tech start-ups are Bla<\/a>ck Swan farming<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

The main point of this note is to suggest that an investor should have a higher tolerance for false positive classifications when selecting shares with right-skew and\/or fat-tailed returns (potential \u201cmoonshots\u201d).  Terminological preamble Statistical discussions of hypothesis testing commonly refer to “false positive” (Type I) and “false negative” (Type II)\u00a0errors.\u00a0The term \u201cerror\u201d is value-neutral in statistical testing, … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-61","post","type-post","status-publish","format-standard","hentry","category-investment"],"_links":{"self":[{"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/posts\/61","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/comments?post=61"}],"version-history":[{"count":4,"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/posts\/61\/revisions"}],"predecessor-version":[{"id":526,"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/posts\/61\/revisions\/526"}],"wp:attachment":[{"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/media?parent=61"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/categories?post=61"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.guythomas.org.uk\/blog\/wp-json\/wp\/v2\/tags?post=61"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}