This post by Hugh Howey originally appeared on Author Earnings with a publication date of 2/12/14. It contains some pretty shocking and encouraging book sales data, at least where indie authors and small publishers are concerned.

It’s no great secret that the world of publishing is changing. What is a secret is how much. Is it changing a lot? Has most of the change already happened? What does the future look like?

The problem with these questions is that we don’t have the data that might give us reliable answers. Distributors like Amazon and Barnes & Noble don’t share their e-book sales figures. At most, they comment on the extreme outliers, which is about as useful as sharing yesterday’s lottery numbers [link]. A few individual authors have made their sales data public, but not enough to paint an accurate picture. We’re left with a game of connect-the-dots where only the prime numbers are revealed. What data we do have often comes in the form of surveys, many of which rely on extremely limited sampling methodologies and also questionable analyses [link].

This lack of data has been frustrating. If writing your first novel is the hardest part of becoming an author, figuring out what to do next runs a close second. Manuscripts in hand, some writers today are deciding to forgo six-figure advances in order to self-publish [link]. Are they crazy? Or is signing away lifetime rights to a work in the digital age crazy? It’s hard to know.

Anecdotal evidence and an ever more open community of self-published authors have caused some to suggest that owning one’s rights is more lucrative in the long run than doing a deal with a major publisher. What used to be an easy decision (please, anyone, take my book!) is now one that keeps many aspiring authors awake at night. As someone who has walked away from incredible offers (after agonizing mightily about doing so), I have longed for greater transparency so that up-and-coming authors can make better-informed decisions. I imagine established writers who are considering their next projects share some of these same concerns.

Other entertainment industries tout the earnings of their practitioners. Sports stars, musicians, actors—their salaries are often discussed as a matter of course. This is less true for authors, and it creates unrealistic expectations for those who pursue writing as a career. Now with every writer needing to choose between self-publishing and submitting to traditional publishers, the decision gets even more difficult. We don’t want to screw up before we even get started.

When I faced these decisions, I had to rely on my own sales data and nothing more. Luckily, I had charted my daily sales reports as my works marched from outside the top one million right up to #1 on Amazon. Using these snapshots, I could plot the correlation between rankings and sales. It wasn’t long before dozens of self-published authors were sharing their sales rates at various positions along the lists in order to make author earnings more transparent to others [link] [link]. Gradually, it became possible to closely estimate how much an author was earning simply by looking at where their works ranked on public lists [link].

This data provided one piece of a complex puzzle. The rest of the puzzle hit my inbox with a mighty thud last week. I received an email from an author with advanced coding skills who had created a software program that can crawl online bestseller lists and grab mountains of data. All of this data is public—it’s online for anyone to see—but until now it’s been extremely difficult to gather, aggregate, and organize. This program, however, is able to do in a day what would take hundreds of volunteers with web browsers and pencils a week to accomplish. The first run grabbed data on nearly 7,000 e-books from several bestselling genre categories on Amazon. Subsequent runs have looked at data for 50,000 titles across all genres. You can ask this data some pretty amazing questions, questions I’ve been asking for well over a year [link]. And now we finally have some answers.

Click here to read the full, lengthy report (including many informative graphs) on Author Earnings. This report should be required reading for anyone who is, or hopes to become, a published author.