Book Analytics

A good novel can be read on many levels. There is always a superficial layer, the story itself. A compelling story can be followed and enjoyed by the widest targeted audience. Beneath the surface, there are often layers of complexity and literary devices at play. Metaphor, themes and satire can be cloaked or revealed transparently. We all learn about this in grade school, and some go further in university really dissecting books for everything the author intended (or maybe didn’t) to present the reader. Here is a slightly, more data-driven way to dig into a book. I loaded the entire content of both On Swift Wings and Gulliver’s Travels into a data analytics workflow to compare and contrast the styles and contents. A few tools used here include sentiment flow, word correlation, word complexity and vocabulary. There are some fascinating details that can be revealed. I hope you’ll enjoy this data analysis of these two novels.

By the way, the script I wrote takes about 5 seconds to run once I have the manuscript, whether from Project Gutenberg or a text/word file. If you’d like to see the same analysis about your book, or a favourite public domain book, just let me know.

Comparing Sentiment Flow

Sentiment Flow Analysis in On Swift Wings
Sentiment Flow Analysis in Gulliver’s Travels

I think these two graphs are particularly interesting. The top two bar charts are an analysis of sentiment value in On Swift Wings (my book) and the bottom two are for Gulliver’s Travels. You’ll note that I’ve blocked out the end of On Swift Wings. I don’t wish to spoil any surprises about whether the ending is happy or sad.

For background, the BING model determines a raw count of whether a word should be deemed “Positive” or “Negative.” Simply put, if the bar is above the line, then the corresponding 1% of the book has more positive words than negative ones. The AFINN model scores different words according to whether they are very positive, somewhat positive, somewhat negative, or very negative and assigns a value that way. In this way, the AFINN model measures the use of emotions with strength. Words like “Torture” and “Ecstasy” bear a greater weight than “Good” or “Bad.”

The first interesting finding is that in general I use quite a few more negative words than Jonathan Swift. The overall balance in terms of raw scores flows from positive at the beginning of the novel to more negative in the later stages. Swift tends to be more positive throughout, in fact, using more positive words particularly near the end of Gulliver’s Travels. (Note, I’m still not talking about the actual end of On Swift Wings.)

While I tend to use more negative words than positive, by weight (AFINN) On Swift Wings has a similar weighted score to Gulliver’s Travels. Most parts of the book are positive, and to a similar degree to Gulliver’s Travels. I think this is particularly interesting. Evidently, I use stronger, more impactful words to counterbalance a general negativity.

Sentiment Word Maps

On Swift Wings Sentiment Cloud
Gulliver’s Travels Sentiment Cloud

To the point about the strength of words used, these word clouds illustrate for each book how commonly different words are used that carry sentiment (size of font) and how impactful that sentiment is (lighter colour = less impact). Both books show many similar words (Great, Like, Good, No, Dead), but there are differences. There are a greater quantity and distribution sentimental words used more frequently in On Swift Wings. Both images were generated using the same code, the difference in shape is due to a difference in style. I invite you to look at the words and compare them yourself. I could look at these two figures for hours.

Again, if you’d like to see your favourite public domain novel, let me know, I’ll run the script and send the results. (I’ll probably put the code on GitHub soon too)

Word Correlation Map

On Swift Wings Correlations
Gulliver’s Correlations

These two figures demonstrate word combinations. Words that are used frequently together are connected. The more often, the thicker and brighter the line. Again, many differences can be seen between the two works. I tend to use a few words together frequently while Mr. Swift has a few clusters of interconnected words, and few other patterns he repeats.

Word Summary Statistics

Measure On Swift Wings Gulliver’s Travels Comparison
Word Count121,426104,280116%
Unique Words11,8488,359142%
Unique Word Ratio9.758.02122%
Average Word Length6.396.21103%

Here’s a really quick little analysis counting the number of words, how many of them are unique, what the ratio of unique to total words is and average word length. It isn’t a valid measure of quality, but On Swift Wings is 16% longer than Gulliver’s Travels, there are 42% more unique words in On Swift Wings, and each word is on average 4% longer. Reading On Swift Wings, you’ll encounter a new word approximately 22% more frequently than reading Gulliver’s Travels.

Before the hate rains down, please remember that this is all good fun. Gulliver’s Travels is a great book, and I strongly recommend it. I only hope that On Swift Wings will be intriguing and entertaining as well.

Weekly Review Section

Thank You Stewart Adams

I received my first review on Amazon this week! As hoped, the book is a challenging but rewarding read. Please keep the reviews coming! Amazon.ca or Amazon.com, Goodreads, Indigo. Reviews are desperately needed to spread the word and get the book in front of more readers. Please.

An interesting modernization of Gulliver’s Travels. There are some great concepts in the book including “perfect” societies and how one person can make a difference.
It is not an easy read due to the meaty sentences, but I am glad I read it.

Worth your time. Stewart Adams – Amazon Review

Cash and a Cold Start

It has been an interesting couple of weeks. My book has now been out for just under three months. This means that I’m starting to get my first royalty payments. In a typically convenient moment, during a span of two hours today, I ran into two things related to the top of my mind issue I’m dealing with right now. (Reviews – Please Review On Swift Wings)

  • The first relates to a favourite cartoonist of mine, Brian Gordon, who is releasing his third book shortly. If you’re a parent, I guarantee that you’ll find his work funny. I’d definitely recommend his books. He posted about the importance of pre-orders for a struggling author. Getting pre-orders helps deal with my second related event.
  • The second came while I was working on a data science course as part of my other job, the one that keeps me from struggling. It was talking about recommender systems like those used on Netflix and Amazon, and the “Cold Start” problem, where until an item has a certain number of reviews, and a sufficient number of people have commented, recommender systems are generally incapable of recommending an item.
Cold Start
Thawing out the cold start

Anyway, I’m trying to figure out an incentive to get reviews online that doesn’t fall foul of the rules and regulations put forward by Amazon and co. I’m not allowed to buy reviews or have family review it, and I don’t intend to risk it.

The other cool thing as mentioned previously is that I got my first royalty payments this week. This is for the few pre-orders that I did receive. Since I didn’t really try to drive pre-orders on my first book, I didn’t expect or get many, but it is pretty cool to get a little money. Now I get to watch the money trickle in.

A little update on the Immortals – book #2. I’m now working again on the plan for the book. I had put it down for a couple weeks to focus on other things, but I’m back at it. I currently have about twenty pages of notes. I think I might show how data science-y I am in a subsequent post, demonstrating my tabular approach to planning, making sure that I am handling all of the themes, characters, and plotlines appropriately throughout the novel. I’ll also show some of the natural language analysis I did of the first book when it was getting close to completion, as compared to Gulliver’s Travels, particularly around sentiment analysis.

Coming Soon: Data Science and Novel Writing

Little Library and Big Bookstore

Free Little Library in Comox, BC – The Bookaneers (On Swift Wings is on the top shelf in the middle)

A short little post on this Tuesday morning:

One fun (and challenging) part of the journey has been to get the book out so that people are aware of it. I really appreciate everybody who has shared the word of my book, and especially recommended it to others. I found this cute free little library in Comox, British Columbia with a couple copies of my book (thanks mom and dad.)

I got a little traction with Indigo books as well. You can now order my book online through their website:

Indigo Hardcover – $39.95

Indigo Paperback – $22.95

Kobo eReader Version – $9.89

They are also talking about ordering copies for a couple large format Calgary stores. (I will definitely be taking a picture if/when that happens.)

I love seeing people with a copy of my book. Please reply with a picture holding the book and where you are located. I have a couple craft-type ideas that I’d like to try and do if enough people respond with pictures.

I’m also talking with Indigo about doing a book signing in the fall at Crowfoot and/or Chinook Centre in Calgary. More details to follow. Thanks for following things. I’ll be working on my second novel in the next couple weeks. With a little luck, the planning stage will be nearly complete and I’ll be writing by September.

BW

Going it alone (with freelancers)

July 31, 2019

Image result for freelancers
Hiring service providers brings its own challenges and rewards.

Once I had decided that I was going to self-publish, a whole new world opened up. Without a traditional publisher, and keeping away from vanity publishers, I would need to research, find, hire, manage, and pay each of my various contracted freelancers myself. I have mentioned before that I really didn’t know anything about publishing at the start, and totally overestimated my own abilities. I’m OK with that, I’d rather think I can do more than I can and pull back when I realize my own limitations than to underestimate myself and achieve less than my potential. At basically every stage where I ultimately hired a service provider, I experienced the same or a similar process.

Stage 1 – I will do it myself!

Original plans called for me to do everything. I was going to plan, write, edit, and format the book. I was going to draw my own cover art and market the book on my own. Tag me as arrogant, but I wanted to try all of these, and I felt like I could do a good enough job if I tried hard enough. I am ok at all of them, but I definitely didn’t know what I didn’t know.

Stage 2 – Research requirements

Image result for book research clipart
Self-publishing requires a lot of research.

Unlike a school project where a teacher tells you all the things that need to be included, there is no definitive requirements document explaining what you need to do to properly edit a book, format the interior, and design a cover. Even now that I’m done, and properly understand the end requirements from experience, I’m not sure that I could properly do any of these myself. Anyway, I researched each one before I gave them all a first try. A blessing and a curse of writing is that the other people who are experts are also writers. They like to write, a lot. There are a lot of resources out there (like this blog is turning into) of people who are eager to tell you everything about their experience. Many of them have written books about how to write books and have them published, I’m not planning on doing that, but rather than a calm stream of helpful advice it is a bit more like a tidal wave of information. Once I had found her, my editor helped to filter some of the more important things. I can’t emphasize how valuable it is to have a guide on this travel.

Stage 3 – Try and try again

I edited my book several times myself, checking for grammar, consistency, idiocy, structure, content, etc. I made several book covers, sought feedback on my Facebook page, edited, and refined them. I formatted my own document using a template I bought online. I took each part as far as I could manage. I did my best, and I felt I did OK, but not good enough. I resolved early on that I would not let myself say, “imagine if I had actually…” like an athlete who sabotages himself before a big race. I wanted to run right through the finish line. The right way to do that is to get the correct people on my side.

Stage 4 – I shouldn’t do it myself!

I concluded that I couldn’t get it sufficiently edited for a professional product. I came to the same conclusion that for a good book, a properly created cover was necessary. With the formatting, I felt confident that I could put a reasonable product together on my own experience, and I think I did come up with something passable. I trusted my editor at this point, who suggested that proper formatting would be a worthwhile investment.

Stage 5 – Research the freelance market

There are a lot of freelancers out there in every field. Whether for editing, cover design, formatting, etc. I wanted to find people compatible with my communication style, who were interested in my book. I did a lot of research, I searched on Editors Canada for Calgary-based editors and found a few people to talk to, I polled my network to see if they knew anybody and drummed up a couple names from there. For the cover design, I considered doing a design competition, but pulled back on that idea when it became clear that experiences there varied dramatically, with all parties generally being dissatisfied with the result, both the artist who feels undervalued and the buyer who doesn’t feel like they got what they wanted. It also hardly feels fair to ask hundreds of people to do the work and only have one or two get paid anything for their troubles. Ultimately I found a few people online who had done a number of covers that I quite liked. Formatting was easier, my editor recommended someone she had worked with and I trusted her judgement.

Stage 6 – Contact

Image result for email clipart

With the exception of the formatting, which was done on my editor’s advice, I had the same thing happen with both the my choice of editor and book cover design. I contacted several people about doing my editing and my cover design. In both cases, the person who I had thought would be my likely first choice rejected the contract.

The first editor to whom I spoke said that I had done a great job of staying very true to the original in terms of tone and voice, and that her editing style was better suited to a close narrative distance. The second editor, who I ultimately picked, was Bobbi Beatty of Silver Scroll Services. When I got in touch with her she wrote me a novel of advice even before I hired her. I felt like she was thoroughly engaged immediately, and I liked her communication style. She seemed interested and excited. I told my wife that I think I had found my editor.

A similar thing happened with the cover design. I found a group that I thought had done some really great covers, but when I contacted them with my book and idea they told me that because my book included ‘fantasy’ worlds and characters, they would be unable to source appropriate images. I don’t hold any malice about this, they didn’t understand my book, and obviously weren’t familiar with Gulliver’s Travels. I pushed on and found a great team at JD&J Book Cover Design. They responded virtually immediately, gave me a timeline and details, took my information and ideas. As with Bobbi, I felt confident with these guys that I had the right people for me.

Stage 7 – Contract

This was pretty easy in every case. Each of my team requested some percentage of the contract up-front. I wanted some details like non-disclosure agreements to protect my own work. (As with most first-time authors, I was unreasonably afraid that somebody was going to steal my ideas.) Each of them provided me with the cost up-front, the timeline within which they would get the work done (and all of them succeeded in completing on schedule) and communicated with me any issues or concerns.

Stage 8 – The Waiting Game

Hungry hungry hippos!

Oh man this is a hard part. Sending something I’d worked on for so long to the editor, formatting, and even book cover designer was like sending my kid to their first summer camp… which I haven’t done yet, so I can only imagine. I told each of my team that I wanted to be the model client. I promised to pay them immediately upon receiving the invoice, to respond promptly to any questions, and to not pester them. I asked them to take their time and do a great job, rather than worry excessively about hitting deadlines. I meant it! I even said that if the book was bad, to just tell me that. I meant that too!

Once the manuscript was out of my hands, I couldn’t do anything about it until it was returned. Every day felt interminable. I waited with bated breath, unsure, and lacking confidence that somebody wouldn’t come back and say, “your book is awful.” Even though I had told them not to worry about deadlines, I still had them circled on my calendar hoping that they would meet them anyway. (they did)

Stage 9 – Final refinements

At each stage, there is always a little back and forth as some details are touched up. A little font change here, a couple last questions about the edits, a few details on the cover, etc. As I had chosen people with whom communication was easy, these refinements went smoothly and quickly. As well as I could have hoped. Ultimately, I was very pleased by the results.

Stage 10 – Payment

promised I would, and I believe that I did pay each of them within a couple hours of receiving their invoice. I know it is important to be paid promptly. One way to show respect and appreciation is to pay quickly.

Stage 11 – Thanks

Really mean it!

I can hardly emphasize how valuable it is to find the right people for your team. I wouldn’t have nearly the final product today that I’m proud to say is mine without all their help. They were worth their weights in gold.

Bobbi, my editor, was my MVP. Beyond cleaning up my rambling, comma-soaked, first-timer manuscript, she provided the guidance and feedback that I increasingly realized I desperately needed. Her ability to match her edits to the tone and voice that I desired was fantastic, her comments and direction were right on point, and her attitude was brilliant. I would strongly recommend her, and when my second book is complete, I will certainly get in touch with her again.

Publishing

July 23, 2019

Image result for gutenberg press
Definitely not the press used…

I’ve written before that when I started writing On Swift Wings, it was purely for the fun and challenge of writing a story. As it progressed, I started thinking more about publishing the book. This was my first foray into the writing space, but I had done a few things that required a little special attention:

  • Voice: I wrote the story in a very formal and classic voice, in as close a style to Jonathan Swift as I could. I borrowed some of his conventions and styles. This also meant that it has a very distant flavour. Most modern novels are written with a very close, in-the-action, sort of voice.
  • Language: I wanted to write a book that was challenging and deep. This definitely won’t appeal to the broadest audience, but it was more important to me to write something of which I would be proud than something that would sell a billion copies and make me fabulously wealthy… that would be cool though.
  • Message: As I laboured on the book, I realized how important it was that I use the platform effectively and actually say what I wanted to say. It was critical to me that the message not overwhelm the narrative, but I wanted to say what I wanted to say.

Originally I wanted to get the book published by a big 5 publisher (Hachette, HarperCollins, Macmillan, Penguin Random House and Simon & Schuster), get the book as widely distributed as possible, and get it out to the widest audience possible. While writing the book, I was researching agents, publishers, subjects, page limits, suggestions, advice, and everything else I could imagine. Obviously, the chances of being a first-time, unknown author getting picked up by a giant publisher is extremely low, virtually impossible.

I explored everything openly, trying to decide what I wanted to do. I would divide publishers into three distinct groups:

  • Classic/Traditional Publishers: Wide, classic, distribution. Unlikely to read a book directly from an author, requires an agent to read the book and sign on first, and then the agent shops around the book to publishers. There are a number of big benefits to classic publishing houses, probably the biggest being the highest potential for sales, and being guided by an expert team through editing and marketing. The drawbacks are significant as well: loss of creative control, huge delays in getting published, smaller share of royalties. Considering my message and voice were important to me, I believe the work and compromises required to work with a traditional publisher would not be in my best interests.
  • Vanity Publishers: I didn’t know anything about this space before I nearly fell into what I now consider to be a bit of a trap. At first glance, these look like traditional publishers except that vanity publishers are like full-service self-publishers. They offer a-la-carte services like editing, marketing, cover design, etc. They will print your book and help you put your book in stores, as I understand anyway. The services are expensive and while it would be helpful to have somebody who knows what they are doing guide me through the process, I felt that in today’s world, I could try and to the job myself.
  • Self-Publishing: As I stumbled on this, led in part by my editor, I felt most comfortable with this option. Self-publishing for me involved publishing on IngramSpark and Amazon Kindle Direct Publishing. KDP is really simple to set up and gets your book up to Amazon almost instantly. IngramSpark is a little more complicated, but it goes out to book sellers, libraries, and online and produces Hardcover books. Choosing self-publishing meant that I would be responsible for editing, covers, formatting, marketing, and all pricing and distribution decisions. While I wasn’t really prepared of this, the internet is an incredible resource, and as I joining a community of people who like to write… well, they all write about everything. A secondary benefit of both of the self-publishing platforms I chose are that they are on-demand. When you put an order in with them, they print a copy of the book at that time. I don’t have to order 500 books and live in a house full of books that I haven’t figured out how to sell.

I chose to go with self-publishing. When you order a copy of my book it is printed on-demand, if you order a paperback from Amazon, it is coming from KDP. If you order either a hardcover or a paperback from somewhere else, it is being printed by IngramSpark.

I chose to go with self-publishing. When you order a copy of my book it is printed on-demand, if you order a paperback from Amazon, it is coming from KDP. If you order either a hardcover or a paperback from somewhere else, it is being printed by IngramSpark. In both cases, printing can occur anywhere in the world. In fact, Amazon apparently uses IngramSpark for their overflow anyway. That way, you can be sure that your copy will arrive as quickly as possible.

Image result for ingramspark

It certainly wasn’t the easiest path, I’ve had to learn a lot about things that I didn’t even know existed before I started. Pricing, marketing, editing, book cover design and requirements, page layout, fonts and formatting, publishing, ordering, taxes, ISBN information, exclusivity, etc. It is pretty amazing the work that goes into every book you’ve ever read. It does play into my own brand of crazy. (I want to understand everything) Learning a plethora of things about diverse subjects was a pretty awesome experience.

A second part of who I am is that I love to tell anybody who will listen everything I know. I swear this isn’t arrogance, but a sincere wish to help everybody be the best they can be. I would be more than happy to answer any more questions about this. Let me know about what you’d like to learn more and I’ll answer as honestly and thoroughly as I can.

Next: Going it alone (with freelancers)