On Empirical and Scientific – vs. Un-Empirical and Un-Scientific – Film Story Research

If you wanted to do an un-empirical study of movies, you could just pick any random films you like, and study them.

But if you wanted to use an empirical dataset, you might select these ones:

(1) The top 20 RoI Movies.

(2) The bottom 20 RoI Movies. (or: `Biggest Money-Loser’ Movies)

(3) And also, 20 `Control’ movies. (i.e., Movies released at the same time, and under similar circumstances, with similar cast & crew, etc, as the Top 20 RoI Movies.)

The top 20 RoI movies means: films that had the lowest production budgets – but the highest audience reach, or in other words, films that were comparatively, the most viral memes.

There are around 500,000 movies in existence (see: Vogel 2011).

So – we could look at the `far-right tail’ in the graph below (i.e., The Top 20 RoI Movies of all time) – and – also, examine the far-left tail (the Bottom 20 RoI movies of all time, i.e., those movies with, the biggest production budgets, but the smallest audience-reach – or in other words, the least-viral memes, as movies)…

StoryAlity Theory - Comparing the Top and Bottom 20 RoI Films (Velikovsky 2014)

StoryAlity Theory – Comparing the Top and Bottom 20 RoI Films (Velikovsky 2014)

Having worked in the domain of Movies for over 20 years, I personally think this is a good way to approach solving one real-world problem for screenwriters and movie creatives, namely: the Less Than 1% Problem in the Domain of Movies.

And I also believe, it’s important to do consilient research – or Science meets the Arts.

For, as Einstein noted:

Einstein 2

“The greatest scientists are always artists as well.” – Albert Einstein

I would even like to suggest that, the vice-versa is also true:

“The greatest artists are always scientists as well.” – JT Velikovsky

I see that the great Neils Bohr noted that the opposite of a great (and profound) truth, is another great truth. (Oscar Wilde also made a fairly decent living out of using that same trick, but for humourous effect.)

Yet in thinking of consilience, we may note how filmmakers like say James Cameron, George Lucas, and Peter Jackson have always been involved deeply in the technical, and technological areas of filmmaking.

All of them have pioneered cinema (both visual and sound) techniques, and have created new technologies: Cameron, with special effects company Digital Domain (such as: the water-weenie from The Abyss (1989) and co-designing submarines for Titanic (1997), and motion-capture and 3D techniques with Avatar (2009) as well as creating the Na’avi language; Lucas with THX Surround Sound, Industrial Light and Magic, and film projection standards as well as anthropology (the monomyth) in Star Wars (1977); Jackson with WETA Digital – and camera technology and visual effects in The Frighteners (1996) and also the Tolkein adaptations (2001-2013).

These successful filmmakers are all technological innovators who routinely use and do science, they are not merely `artists’ as film storytellers.

From The Structure of Scientific Revolutions (Kuhn 1962, 2012):

`…those are also the years, particularly during the Renaissance, when little cleavage was felt between the Sciences and the Arts.

Leonardo was only one of many men who passed freely back and forth between fields that only later became categorically distinct.

(Kuhn and Hacking 2012: 160)

Leonardo da Vinci stars as: Gandalf da Gray

Leonardo da Vinci stars as: Gandalf The Gray

The Arts – and the Sciences – are usually seen as separate domains.

But – are they?

StoryAlity and Storyology - at the intersection of the Arts and the Sciences

StoryAlity and Storyology – at the intersection of the Arts and the Sciences

Doesn’t creativity work exactly the same way in both the Arts, and Sciences?

i.e. And – didn’t Csikszentmihalyi (and along with DK Simonton, one of the current leading world experts on Creativity, in the domains of Psychology, Sociology, and Culturology or Memetics) empirically demonstrate this?

Csikszentmihalyi has more to say about: “the Arts versus the Sciences” – and – about creativity actually happening in both:

`What makes [the] breakdown in communication among disciplines so dangerous is that… most creative achievements depend on making connections among disparate domains. The more obscure and separate knowledge becomes, the fewer the chances that creativity can reveal itself.’

(M. Csikszentmihalyi 1996: 338)

Also – what else did Csikszentmihalyi say about creative problem-finding and creative problem-solving:

`The goal may emerge as a problem in the Domain – a gap in the network of knowledge, a contradiction among the findings, a puzzling result.’

(M. Csikszentmihalyi 1997: 114)

(Is it puzzling to anyone else that, 7 in 10 films lose money? And that – none of the screenplay/story gurus use an empirical or scientific methodology? In fact, that most have no `methodology’ at all? Might this possibly explain why: 7 in 10 films lose money? Because of: The Film Story?)

Also stated in the same 1996 text, by Csikszentmihalyi:

`Your research project gets defined partly by some internal fascination for which one cannot account in any detail, preparation that is unique because of the life history of that person, luck, and something to work against. That is, something that you are dissatisfied with, that other people are doing.’

(M. Csikszentmihalyi 1996: 88)

Also… Apart from the above – “puzzling results “- Is anyone besides me sick of the product that Hollywood now serves up, as movies?

McKee spots a film turd

…Is anyone besides me (speaking as a working Transmedia screenwriter) finding it scandalous – that the current “screenwriting convention” results in 7 in 10 movies losing money?

7 in 10 feature films lose money

7 in 10 feature films lose money

(And… Has anyone else noticed that, all the other domains threw away Aristotle’s ideas, from 2300 years ago, about 300 years ago, at the latest?)

At any rate:

On Empirical vs Non-Empirical story research


What exactly is Empirical and Non-Empirical research in story/narratology/film?

Simonton states in Great Flicks: Scientific Studies of Cinematic Creativity and Aesthetics (Simonton 2011)

`Many of those involved in movies – whether filmmakers or aficionados, journalists or critics – operate according to an explicit or implicit assumption: It is possible to identify something that might be called a “great flick”…

Oddly, with the exception of  box-office success, comparatively little scientific research has been dedicated to addressing this issue. In drawing this unfortunate conclusion I must emphasize the adjective scientific. So before we can continue, it is first necessary to describe what I mean by this term.

Science – Most of the research in film studies originates in the humanities. The contributions consist of insightful essays that examine a diversity of questions. Far less common are investigations that can be considered scientific, in a strict sense.

By “scientific” I mean a study that is abstract, systematic, objective, and quantitative.’

(Simonton 2011: 5-6)

Simonton also says:

`The sampled films in these studies are not randomly selected from some indefinite population. Instead the films are systematically selected according to precise selection criteria.

Any researcher who collects a sample of films employing the specified criteria will obtain the exact same films!

(Simonton 2011: 40)

This is also the case with this doctoral research study of the top 20 and bottom 20 ROI films. See the data set here:

The-Numbers.com http://www.the-numbers.com/movies/records/budgets.php )

And the final data set for the empirical research study:

The Top 20 Audience Reach/Budget Films of the Last 70 Years. Data Source: The-Numbers.com. Analysis: JT Velikovsky

The Top 20 Audience Reach/Budget Films of the Last 70 Years. Data Source: The-Numbers.com. Analysis: JT Velikovsky

Note that: I have collated the films of the past 70 years, excluding those films older than this timeframe.

(You can use Wayback Machine to find the older films.)

Notably, as it happens – the 3 older films excluded from the study sample set also follow the high-ROI StoryAlity story pattern.

So then, why exclude them from the empirical study?

Because their other characteristics – such as running time – `skew the data’ in terms of where cinema has evolved to, currently.

The `modern cinema era’ began, in terms of our data sample – in 1968 – with Night Of The Living Dead. Prior to this, cinema was even more dominated by “the big film studio era” and the structure of the industry was very different, for working filmmakers.

Therefore, studying those 3 very old films (Birth of a Nation (1915), The Big Parade (1925), Gone With The Wind(1939)) is counterproductive for our purposes: to find out what makes a film story go most viral, now.

(While film historians, critics, and The Academy may of course find it very useful and interesting data, but: for a currently working screenwriter/filmmaker – it is not practically useful.)

Returning to Simonton, in Great Flicks: Scientific Studies of Cinematic Creativity and Aesthetics:

`Screenplay Attributes – Because of the complexity of the published findings, I concentrate on those results that have received the most attention in scientific research. In particular, I scrutinize the following attributes:

(a) the screenplay’s genre or general story type,

(b) its final running time on the silver screen,

(c) its MPAA rating during its theatrical release,

(d) the type and intensity of “adult content” depicted in the film,

(e) whether the script has connections with prior films as either a sequel or a remake,

(f) whether it purports to tell a true story about a real event or person, and

(g) whether it is an adaptation of previous narrative material that has been successful in another medium besides feature-length film.

In the last case we’ll also consider the specific nature of the adaptation and who carried it out.

Each of these script characteristics will be discussed with respect to three criteria: financial performance, movie awards, and critical evaluations. I’ll also include production budget in the first criterion so as to shed additional light on the distinction between films as art and movies as business. As we’ll see, there are really two major kinds of screenplays.’

(Simonton 2011: 81)

To be clear, this is the data set of my empirical doctoral research study:

The primary data set:

The Top 20 Audience Reach/Budget Films of the Last 70 Years. Data Source: The-Numbers.com. Analysis: JT Velikovsky

The Top 20 Audience Reach/Budget Films of the Last 70 Years. Data Source: The-Numbers.com. Analysis: JT Velikovsky

And – The Secondary Data Set: The Bottom 20 ROI FIlms (or: “the biggest money losers” to date)

Bottom 20 ROI Film List

The Bottom 20 ROI Film List

By comparing and contrasting these two data sets, we can see what characteristics the most viral film stories have. (i.e The Top 20 ROI films and the Bottom 20 ROI films.)

The Film ROI (return on investment) bell curve

The Film ROI (return on investment) Gaussian bell-curve

Note: This data is an empirical sample. – The data set `selects itself’. It is not composed of films selectively or illustratively chosen by the researcher (i.e. – JT Velikovsky). The list `is what it is’. Anyone researching the top 20 RoI films will find themselves studying the same set of films. Interestingly, they have over 30 things in common.

Those other sorts of non-empirical studies of film/screenplays are: not useful. They won’t necessarily tell you how to make a successful film: just – how to make `a‘ film.

For example, McKee in Story declares his absence of methodology in his “research” right up front – in the Notes On The Text section of his book Story:

“The hundreds of examples in Story are drawn from a century of film writing and filmmaking around the world. Whenever possible I offer more than one title of the most recently and widely seen works I know.

Because it’s impossible to select films everyone has seen and remembers in detail, I’ve leaned towards those readily available on video.

But first and foremost, each film has been chosen because it is a clear illustration of the point made in the text.”

(McKee 1999: Notes On The Text – my emphasis)

Therefore, McKee has very clearly admitted – his “research” is not empirical – so, are his “principles” simply his own invention? – Are they result of his selective observations, chosen to illustrate his theories/`principles’?

And – where did those `principles’ come from? Aristotle, from 2200 years before film was invented..? Do those “principles” have any supporting empirical evidence, now? (Also – Did Aristotle ever actually say anything about “3 Acts” , anyway?)

This lack of a scientific or empirical method is deeply problematic, when presented as “research” or even “knowledge”- as, it reveals a Romantic (mystical, non-scientific), and not a Rationalist (scientific) view of Creativity. (For a rationalist view, see: Csikszentmihalyi, Simonton, Sawyer, Koestler, Boden, et al. Or – read all the posts in this blog, from #6 through #14.)

And in direct contrast to McKee (and other screenplay `gurus’) – DK Simonton uses the (excellent) “compare the top and bottom data sets” methodology – as does this doctoral research – in his truly excellent and admirable work, Great Flicks: Scientific Studies of Cinematic Creativity and Aesthetics (2011):

`If the Oscars and other honors help us comprehend the upper-right tail of the distribution, why can’t the Razzies aid our understanding of the lower-left hand tail?

The negative honors allow us to round out our knowledge of the full range, from turkeys to masterpieces.’

(Simonton 2011: 177)

It is for these reasons – primarily the research methodology utilized, including the scientific methodology of Creative Practice Theory Narratology, that this research study is a valuable new contribution to knowledge.

A synthesis of theoretical lenses for examining creativity in the domain of movie story (Velikovsky 2014)

A synthesis of theoretical lenses for examining creativity in the domain of movie story (Velikovsky 2014)

For more on consilience (using science in the arts, and – why it is now more important than ever before) see this post: StoryAlity #75 – Consilience in The Study of Film – and the Arts.

And for more on Creative Practice Theory, see also: Creative Practice Theory.

And thanks for reading!

– Thoughts/Comments most welcome.


JT Velikovsky

High-RoI Story/Screenplay/Movie and Transmedia Researcher

The above is (mostly) an adapted excerpt, from my doctoral thesis: “Communication, Creativity and Consilience in Cinema”. It is presented here for the benefit of fellow screenwriting, filmmaking and creativity researchers. For more, see https://aftrs.academia.edu/JTVelikovsky

JT Velikovsky is also a produced feature film screenwriter and million-selling transmedia writer-director-producer. He has been a professional story analyst for major film studios, film funding organizations, and for the national writer’s guild. For more see: http://on-writering.blogspot.com/



Csikszentmihalyi, Mihaly (1996), Creativity: Flow and the Psychology of Discovery and Invention (1st edn.; New York: HarperCollins) viii, 456 p.

— (1997), Creativity : flow and the psychology of discovery and invention (1st HarperPerennial edn.; New York: HarperPerennial) 456 p.

Kuhn, Thomas S. and Hacking, Ian (2012), The Structure of Scientific Revolutions (4th edn.; Chicago ; London: University of Chicago Press) xlvi, 217 p.

McKee, Robert (1999), Story: Substance, Structure, Style, and the Principles of Screenwriting (London: Methuen) 466 p.

Simonton, Dean Keith (2011), Great Flicks: Scientific Studies of Cinematic Creativity and Aesthetics (New York; Oxford: Oxford University Press).


8 thoughts on “StoryAlity #36 – On Empirical and Scientific vs Un-Empirical and Un-Scientific Film Story Research

  1. Pingback: StoryAlity #53.2 – Patterns in the Top 20 ROI Films: # of Scenes vs Film Duration/Screenplay Length – and: Elliot Waves | StoryAlity

  2. Pingback: StoryAlity #56 – Patterns in the Top 20 ROI Films: # of Scenes vs Film Duration/Screenplay Length – and: Elliot Waves | StoryAlity

  3. Pingback: StoryAlity #60 – On Character and Story – in the Top 20 ROI Films | StoryAlity

  4. Pingback: StoryAlity #9B – Creativity in Science (and – The Arts and Film) | StoryAlity

  5. Pingback: StoryAlity #66 – Scientific and Empirical Proof that the StoryAlity™ Theory works: `The Devil Inside’ (2012) | StoryAlity

  6. Pingback: StoryAlity #76 – The `new’ Film RoI numbers at The-Numbers.com | StoryAlity

  7. Pingback: StoryAlity #84 – Pinker on `Art and Adaptation’ | StoryAlity

  8. Pingback: StoryAlity #5B – Almost Everything You Know About Movie Screenwriting is Wrong | StoryAlity

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s