Regarding Google’s advice to learning app designers

There is a growing public perception that “most educational apps stink” in today’s App Store, in part because they are ineffective.  That’s partly why I’m so happy to see Google promoting quality apps in their new App Store for Educators:

“Apps submitted to Google Play for Education will be evaluated by a third-party educator network, which will review them based on alignment with Common Core Standards and other factors.”  In the demo video, it is revealed that CUE is the 3rd party doing the reviewing.

I’m also very happy to see Google offering design advice to educational app designer/developers.  In this article I suggest ways Google could improve that advice.

In this first section, I argue that Google should require app developers to prove their app is effective.  I then review Google’s advice more broadly.

If I could make only one change…

If I could make only one change to this list, I would add this:

  • Prove your app is effective.

For example developers should be required to say, “Students who played the game [Motion Math] for 20 minutes for five days improved on a fractions test by an average of 15%.”  (link).  Pearson offers a free, generic framework (link) and many other similar resources exist.

I’m not talking about screening low quality apps.  I’m talking about screening apps that don’t measure anything at all.

Google told learning app designers (here, my bold):

Apps with highest educational value will have these characteristics:

  • Designed for use in K-12 classrooms.
  • Aligned with a common core standard or support common-core learning.
  • Simple, easy to use, and intuitive for the grade levels the app is targeting. App is relatively easy to navigate without teacher guidance. Not distracting or overwhelming to students.
  • Enjoyable and interactive. App is engaging to students and lets them control their experience.
  • Versatile. App has features make the it useful for more than one classroom function or lesson throughout the school year.
  • Supports the “4Cs”:
  1. Creativity — Allows students to create in order to express understanding of the learning objectives, and try new approaches, innovation and invention to get things done.
  2. Critical thinking — Allows students to look at problems in a new way, linking learning across subjects and disciplines.
  3. Collaboration — Allows students and (if appropriate) educators to work together to reach a goal.
  4. Communication — Allows students to comprehend, critique and share thoughts, questions, ideas and solutions.

Edutainment, initially hailed as a educational revolution, failed to disrupt classroom practice. One of the many reasons, argued MIT researchers, was the products’ frequent lack of efficacy (link). Google could help the latest generation of developers avoid repeating this clear and well-understood mistake in the field.

Bad learning apps can actually hurt learning. Some popular learning products are widely believed to be ineffective (such as toddler DVDs), but it is less commonly known that bad learning apps can do harm, not just fail to do good.   “Zimmerman, Christakis, and Meltzoff (2007) empirically demonstrated that for each hour children, ages 8 to 16 months, were exposed to commercially available audiovisual programs (e.g., Baby Einstein and Brainy Baby), the children developed 6 to 8 fewer receptive vocabulary words (i.e., words they understand) than their counterparts who were not exposed to such stimuli.” (Christakis 2009).  Google should prevent ineffective products from being confused with unknown or proven good educational products.  Requiring any sort of efficacy evidence would be a simple way to screen many of these products.

Obviously not all 1-person app developers can afford to do a “proper” randomized controlled trial, but I believe anyone can do a simple pre-post efficacy test. Some learning goals are less obviously testable. How does one evaluate efficacy of “systems thinking”?  It can be done, if only by using very qualitative, unstructured interviews.

I wish Google should require all apps in the education store to

  1. make a clear, specific claim of efficacy,
  2. provide evidence of that claim, and
  3. have that evidence validated or reviewed by a 3rd party

Google’s CUE approval system is a good first step toward the 3rd point, but I hope for more: I want a scale rating,  not just approved/not approved, so proven apps are first on the list, and the reasons are clear.

A Broader Critique

Next, I want to talk more broadly about Google’s design advice: Is this list good advice? 

Advice is cheap to make, but VERY EXPENSIVE to follow. Every point on Google’s list adds huge cost and risk to the app developer.

Specifically, I ask:

  • How should developers decide which, if any, of these to follow?
  • Should other stakeholders, say publishers, criticize an app, using this advice?
  • How?

There is difference between a wishlist and useful design advice. For example, consider this design advice: A quality car should include as many of these features as possible:

  • seats 12
  • 100 mpg
  • 0-100-0 in 4 seconds
  • Less than $10,000
  • Made from environmentally friendly materials
  • Looks awesomer than a Lambourghini
  • Parks in half a parking space

I hope we can agree that this list is near-impossible for commercial, practicing car designers to adhere to, and that it unlikely to be useful to audience.  Compare that silly list to Google’s list, and note what the two lists have in common, as you read the following questions.

  • Where did this advice come from? Who wrote this? What are their qualifications?
  • Is the source ‘data’ trustworthy?  Is this a wishlist of a naive enthusiast?  Is it based on lessons learned from a single case study? Is it a broad summary of the academic literature, written in an ivory tower?
  • Does this advice apply equally across the entire diverse landscape of the field?  Should learning game apps that practice, be more collaborative than instructional apps?
  • Does this advice fit with other expert design advice?  See below for examples. Are there conflicts or commonalities between this advice and existing, prevailing views of experienced designer/researchers?  What reasons are given for this variance?
  • Is this advice realistic? Is it even possible to build an app that fits all, or even most, of this advice?
  • What are some examples of apps that follow this advice?  Discuss merits and weaknesses of exemplary designs.
  • Could and should this advice be used by stakeholders, other than developers, to assess or critique?
  • Is there any evidence or reason to believe this advice will yield improved learning apps?  Are there cautions on any dangerous combinations?

I hope the reader can, by comparing to the silly list of car design, see why and how Google’s advice might be improved upon.

How useful is broad advice, to 1-person app developers?

What use is design advice for a “car”?  Minivans, supercars, and econoboxes all have very different use cases.  There is precious little design advice that applies to all.

A naive advisor might argue that these traits are all desirable. What’s the problem with advising designers to aim for such traits?  THe problem comes in assuming all learning apps are essentially similar.

Consider how a supercar designer who is told: cars should be affordable. Should they try to make a $10,000 supercar?  Of course not. It would not be possible to meet the key requirements of a supercar (performance, style, etc) in a $10,000 cost ceiling   Why not try to make minivans take half a parking space?  Again, the value of the minivan is its hauling capacity.  A tiny minivan is not a minivan anymore. It’s a different type of car.

Good learning apps are not essentially similar.  Teaching the concepts of algebra has little in common with reviewing cultural norms in 17th century Africa.    Proponents of gamification, applied to cars, suggest we can reuse mechanics for a variety of purposes.  That’s like saying we can all adapted a Ford Taurus to our needs: Farmers can add a roof rack, instead of buying a pickup truck, for hauling brush.  Racers can put chrome rims on and bingo, teen revheads have a cool car.

How many e-learning apps are basicly flash cards?  show material, multiple choice. Such elearning designs can be effective but designers should work hard to improve on that weak interaction. Such designs are not the best we can do with the power of Android apps.  I believe Google offered this advice intending designers to aim higher, as Devlin explains well here.

So, how what should the advice be?  Following the car metaphor, supercar designers should be discussing specifics: the merits of carbon fiber in interior detail, for example.

However, there is need for basic advice aimed at one-person learning app designers who didn’t necessarily study e-learning design principles in school. Such designers are perhaps akin to kit-car builders:

  • They need a few basic ideas (more rubber on the road means more traction, but higher friction). I think this was Google’s intent with this list, and I give some of my favorite examples of such advice at the end of this post.
  • They need many specific tips (e.g. slant your kingpins to make the car steer straight). this is tough to deliver on paper – it needs to be “just in time” and very simple, and pushed to designers as they work.
  • They don’t need broad goals (make your car use less gas). I think Google accidentally delivered much of this type of advice.

There are some general points, such as those made by

Much design advice should be specific to the intended learning goal, age, and nature of outcome (practice, etc).  Learning designers ask:

Should we repeat material?  Is it worth building a proper simulation, or just semi-faking it with a simple 1-variable interactive element?   Where does learning really occur in apps?  How can we collaborate yet avoid the blind leading the blind of the cliff?  There are some clues and a few outright answers in the literature (it’s not very accessable and easy to find, but that’s a separate rant).  That’s the design advice we need.

The end.

PS Further Reading

Finding good advice ain’t easy.  I’ll give three personal favorites, for classroom learning game design.

  1. MIT’s “Moving Learning Games Forward” paper here,
  2. Gee’s numerous excellent principles here (summarized by Draper here).
  3. For math learning games specifically, Devlin’s blog here.

These three examples are specific to learning games, part of the vast literature on e-learning (a random example of which is here).

<whap> Thank you sir. May I have another?

I am considering writing a review where I compare, point by point, Google’s advice to prevailing views from Gee, Osterweil, specifically for learning game designers. (if that’s something you’d be interested to see, let me know).

LegitMe Pitch

Do YOU want to disrupt the world of e-retail? Are you unafraid of Amazon, Apple, Hollywood, and every other powerful IP holder in the world?  OK!  here’s the plan:

NEED 1: CONVENIENCE. I, the LegitMe customer, used to be a pirate but as I get older and richer, I use torrents not to save money but time, and to get a better product. Software is quicker and easier to install via torrent, and DRM-free media is handy and quick, compared to regionlocks and other silliness of legit media. What I want is a way to legalize everything I’ve illegally downloaded on my pc, quick and simple.

SOLUTION:  LegitMe is a client and ASP, endorsed by an organization I totally trust, and that I install voluntarily.  When I run the client, it crawls all storage on all my personal devices, identifies illegally downloaded media (torrents, mainly). Once a year or so, it would invite me to buy all my downloads: “This computer, and WifePc, and Kidspc, together have 258 songs, 32 movies, and 6 apps that aren’t legal. To buy these all for $379, click here. To buy and/or delete certain ones, click here.”  I pay $379 to LegitMe’s paypal account, and I’m done (actually, not quite done – more on this later).

Once I’ve OKed it, the client send parts of that list to thousands of other clients using P2P.  After the data is sufficiently anonymized, the clients upload the list (no personally identifying information) to LegitMe’s servers.   The servers tally how many copies of each media are there, and contacts the rightful owner.   e.g. “Dear MGM: 6271 torrent owners have illegally downloaded “Full Metal Jacket” and have asked us to buy it for them under these terms [link]. Based sales history of similar titles (viewable here), and with an amnesty discount, we believe a fair price is $32,800 ($5.23 per copy).  To agree, click here to receive imeediate, cash payment.  If you do not accept this price or terms, feel free to counteroffer by contacting”


LegitMe doesn’t know and doesn’t promise the user is obeying the terms of the IP deal.  There is no “ownership receipt”.  Angry IP owners can subpoena LegitMe’s servers, but there’s nothing on them that would help identify users. It would be more useful to get torrent download sites, which at least have IP addresses associated with the torrent download.

LegitMe allows the users to set the terms, and forces IP Owners to negotiate on stupid ideas like Region Lock. It’s based on trust.  LegitMe allows honest users to “come clean,” out of the goodness of their hearts.  In their hearts, most people do want to be good.  They want to see indie artists rewarded.  Most even want James Cameron to get $10M to make Avatar 2.  LegitMe allows them to be good without being punished.  Two examples:
1) I’ll pay something for Windows XP, but not $100, and I’m not driving to Best Buy to get it!
2) Collectively negotiating terms could open exciting new business opportunities to indies (“Dear AngryTanks developer: 253 users are collectively offering you $32,393 to open source your entire game”) but would be more often used to counteroffer untenable terms in major IP Owners’ licensing agreements (“Dear MGM: Owners have authorized LegitMe to pay $0.37 more for each of your DVDs if they are not region-locked.”).

RISKS:  I can think of only three teensy little risks….

RISK 1. IP owners like Sony Music are known to be vicious,greedy sharks. Would they really negotiate with known pirates?

A.  They will have no choice.  A key hire for LegitMe would be a really excellent IP licensing negotiator.  At first LegitMe would only be able reach agreement with progressive, openminded IP owners (e.g. O’  It would not even try to negotiate with larger entities; it would simply hold the money (with users’ permission) in escrow for a future date, while it continued to bring on more and more IP owners.  The money would be stacking up, over the years. If/when the escrow reached millions, LegitMe would fund a campaign to pressure the big boys, working on them via stockholders (they are legally obliged to make a profit), competitive pressure (progressive publishers would have a “pure profit” revenue stream), and political (good old campaign finance, seeking a “obligation to offer fair alternative” law).

If LegitMe works, eventually I (the user) might trust LegitMe enough to not go anonymous.  At that point, I can track my IP ownership across multiple devices, download higher quality versions of the content from IP owners, build a genuine relationship of trust with the IP owner directly.

In the eyes of the IP owners, LegitMe would work like a bricks-and-mortar retailer: buying IP in bulk, competing with Apple and Amazon.

2. Would enough users really trust this product not to rat them out?
This is the biggest risk, at this point. Certainly a few people will do it, but also certainly some people would never do it.  A realistic estimation of the market size would require some research.  Partnership or endorsement by a completely trustworthy entity would help.  The EFF springs to mind. Suggestions?

3 Is it possible for code to determine which media you had purchased, as opposed to legally downloaded or ripped content?
LegitMe would allow users to tag IP they bought legally (e.g. DVDs they ripped themselves), versus downloading.  LegitMe would see a very large (90% say) discount for media I bought and ripped, provided I share them only among devices I own.


Obviously it won’t be easy.   And also obviously, there’s a LOT of money in it, if it works. To me that’s a risk, not a goal.  Anything worth $100m gets greedy sharks circling.  We are not sharks.  We are wizards, changing the world with the power of pure ideas.  We must protect this from sharks – hence “social enterprise.”  This idea can rebalance the scales of IP ownership by letting normal people get rewarded for being honest.

To the investor, LegitMe is a disruptive new business model for e-retail that will beat Apple and Amazon three ways:

1) cost of doing business is massively lower – no ads, retail website, etc.
2) licensing terms are set by purchasers, not owners.
3) negotion leverage: IP owners can cut off Apple and Amazon if they wish.  They can’t do that with LegitMe.

Critiques? Suggestions?  I honestly don’t see how this can’t work.  Fire away!

an letter to kids who want to mod minecraft

A smart gamer kid recently emailed me:

> i forgot to say in my last email that it would be epic if you could show me how to make a mod for minecraft!!!

Here’s a public reply, for that kid and all the world of kids like him.


Are you a kid who wants to learn how to mod minecraft?

First: IF YOU CAN MOD GAMES, YOU ARE SERIOUSLY AMAZING!   Modding teaches you really important skills: how to program, create art, and work with the biggest, baddest kind of program ever: video games are hardcore.   So, it’s fun to make them, but it could be more than just fun. It could be a big deal if you take it seriously and stick with it.

The bad news is: it’s not easy!  Minecraft wasn’t designed for modding, so the code is confusing and complicated.  I love Minecraft too, but I would NOT suggest learning to mod on it.  Mod some other games first!  I know some way easier games to mod. Here’s 3:

  • If you can build lego, you can mod games with By playing this game, you’ll learn to mod, and build your own, simple games – WITHOUT having to learn anything hard at all!  You just drag and drop little guys and blocks and click “play”.  SUPER easy.  What you learn here will help you mod harder games.
  • Once you’ve beaten GameStar Mechanic, you will probably want more controls than just a gravity slider and whatever.  You need a “game engine”.  Here’s two: Construct ( and GameMaker ( No coding needed, and you can build some seriously fun games, or just mod the ones it comes with.  (but to build more complex games in GameMaker, you can also write code.
  • If you want 3D games, one really powerful game engine with lots of good tutorials is Unity ( You can build most kinds of games with Unity.

…but I bet some of you will ignore that list because YOU just want to mod MINECRAFT.  I would have said that when I was 12.  OK, OK!

The tutorial below assumes you know how to program (I call programming “coding”).   So, to learn to code enough to mod Minecraft, do this tutorial:   It’s free, fun and easy – no installs or downloads – just watch the videos and code right in the browser.   There’s another site called which is also great.

OK, so you know how to code a bit.  Here’s how to mod minecraft.

1. install something called MCP (instructions here).

2. do this tutorial. It’s step 4 of a longer tutorial. I skip steps 1-3 because they mainly teach some advanced coding ideas, which are confusing for beginners. In step 4, you make a new kind of dirt block, and see how Minecraft really works.

3. Keep going. There’s lots more tutorials here.  Or, you can continue with the “lightdirt” tutorial:

Don’t give up.  Search around for other mod tutorials if you don’t like this one.  Try youtube “mod minecraft tutorial”

Take your time.  If you can do this tutorial, even if you don’t understand it all, you are seriously amazing.  This is like a full university!

You can do this, as long as you don’t give up.  I learned how to program on my own when I was 12 because i was SO KEEN to mod my favorite game, I NEVER gave up. Even when it was frustrating, I just WANTED to mod SO BADLY – I just kept struggling till one day, it worked! I was awesome.

If you get frustrated with modding Minecraft, don’t feel bad!  Modding can be quick, simple, and really fun (not modding Minecraft! it’s hard, complex, and really fun 🙂 ).  Try GameStar Mechanic.  Try Construct.  WAY easier.

When I was 12, I started with WAY easier games than Minecraft.  I probably would have been too frustrated with Minecraft to finish.  My first games were more like Pac-man.  I did really dumb mods of games (and I didn’t try hard games like Minecraft! I started with easy ones first).  Then I did cooler games.  From there, no one could stop me – and now I get paid to design games – awesome!

Also.  There’s two kinds of mods:  art mods, and real mods.  Art mods (skins, levels) is fun, because you can just click around or paint in Photoshop, but for a game like Minecraft, art mods can’t change how the game WORKS.  If you want to really control the game, you’ll need to code.

I would be so impressed if you learned to program properly.  Programming is the real thing.  If you can write a game, you can write code that controls everything – robots, cars, airplanes, figher jets. You can earn good money.  If you want to really control the world, learn to program.  And, it’s really fun!


edge flow reduction – two simple tricks from topology

If a 3d modeler needs to get rid of polygons in a 3D mesh, but don’t want to add triangles, here’s two good ways. I call them “3-to-1” and “2-to-1 corner”.  I worked these out in 1988 with Joe Formicola and Ken Fedeson, building finite element models for CFD and FEA.  Many other 3d artists have discovered or learned these topological tricks, since (and probably before then, too).  I just noticed how the first one resembles the shadow of a hypercube.