TechnologyThe near and far future of emoji
We asked language technologists, developers, and digital media scholars to help us forecast the ever-changing world of emojis, our favorite little presets.
Emoji, n., Japanese from e ‘picture’ + moji ‘letter, character’. Who would have guessed that a pile of poop from the nineties would make its way into the Oxford English Dictionary in 2013 with zero backlash to boot? When cell phone users in Japan began utilizing their device to send pictures, an industry that was already struggling to keep up with 80 million customers knew it was time to innovate. Whereas a picture can be limitless in size (proportional to multiple texts), a standardized image with a preset, single character code is the same as choosing a letter on your keyboard (thusis equivalent to typing out "hey" as far as your phone is concerned). The main difference being, of course, that the interpretations and nuances of digital icons extend well beyond those of the written word.
While these nuances are groundbreaking in an esperanto sense—an eggplant is an eggplant is an eggplant, after all—they also create a huge margin for error and a demand for a larger emoji vocabulary for a quickly evolving future. To help us navigate the smiley-laced times ahead, we consulted the cutting edge in type technology to see what’s in store for a future of keyboard-as-canvas.
CEO and founder of Tavultesoft, the company that created Keyman (software that allows you to type in more than 1,000 languages)
To understand the direction of emoji, it can be helpful to understand how they are input on computers and mobile devices. When there are a handful of emoji or classical emoticons, a picker is a great way to insert a smiley face or a birthday cake. However, the more emoji that are encoded, the less viable this approach becomes. The input method developer is forced into breaking the emoji collection down into categories (which category do you place U+1F4A9 PILE OF POO into?), and the selection of emoji for construction of messages becomes tedious at best.
Some people have compared typing emoji to typing Chinese. In reality, these two writing systems bear very little similarity. Chinese is a logographic script, where shapes represent words, morphemes and phonetic sounds. This basis allows for many logical ways to build an input method. However, shapes in emoji represent concepts. How do you type a concept?
One way to type a concept is to use a mnemonic such as “(cool)” to type U+1F576 DARK SUNGLASSES. However, typically users can remember only a small set of regularly used mnemonics, so this will only then be helpful for a small set of characters. This also constrains the meaning of the dark sunglasses in the user’s mind to “cool” and will they think of “bright sunny day” when someone else uses the same emoji?
This ambiguity inherent in emoji makes it difficult to communicate meaning. A smiley face can be helpful in communicating emotion, but what meaning does U+1F481 INFORMATION DESK PERSON really convey? In order for emoji to be really useful in general communication, the set would have to be continually expanded. The corresponding increase in complexity in typing or selecting emoji suggests that they will remain at most an amusing adjunct to classical modes of communications.
Founder of Tiro Typeworks (digital type foundry founded in 1994 specializing in multilingual computing -- clients include Linotype Library, Agfa and Microsoft), recipient of the Golden Bee (Design Biennale) Association Award for ‘outstanding contribution to the development of Cyrillic typography and international typographic communication’, has a piece in “Language Culture Type: International Type Design in the Age of Unicode” (Graphis Press, 2002), frequent lecturer on typographic issues.
Pictographic communication systems are nothing new. In numerous places around the world, they appear in archaeological evidence as the first instances of human efforts to graphically communicate. What interests me, though, is not their great age — and hence the ancient heritage of today's emoji — but their rapid and universal replacement by graphical systems that represent linguistic content (phonology, morphology, syntax, etc.). I think there are very good reasons why that happened, and these are the same reasons why I don't get excited about the communicative potential of emoji. Simply put, when it comes to communication — when it comes to the always imperfect conveyance of thoughts from one person's mind to that of another — language is by far the best means we humans have at our disposal, and written language lends both longevity and precision to those means (we seldom revise what we speak, but I have revised this written sentence several times).
I'd go further, and suggest that when people talk about 'communication' in terms of anything other than language, e.g. 'visual communication', they mostly do so in metaphorical terms that reference the experience of linguistic communication, and they do so unconsciously because language is so deeply ingrained in not only how we communicate with each other but also how we communicate with ourselves. Our brains are alive, through our senses, to immense quantities of stimulus experience, but it is in language that we construct ideas — construct even the idea of an idea — and through which we make sense of experience. It is in language that we primarily communicate those ideas and share our experience with others, and when we realized that we could capture language in a visible form, we everywhere abandoned pictographic systems: we moved quickly from recording the image of a thing to recording the word for a thing. The systems that humans in different places developed to do so vary richly, exhibiting different degrees of precision and efficiency, but even what might be considered the closest surviving relative of pictographic communication — the logographic scripts of east Asia — also convey phonetic information and rely on strongly conventional relationships to language. This isn't surprising, considering that an important use of writing is to record oral testimony, be it a legal deposition or a poem.
So where does that leave emoji, the pictographic system du jour? Well, the use of symbols has never gone away. They have persisted alongside and in support of linguistic communication, often providing an efficient shorthand for complex constructs, as in symbolic logic, or space-efficient aids to navigation, on a map or in a user-interface. They have always had their place, and emoji — which are not a set of symbols but rather a particular technology for the interchange of non-linguistic content between devices — has its place too. That place isn't necessarily where we think it is, though, or where, for expedience, we've located that technology.
When it was first proposed to encode emoji in the Unicode Standard, I was among those who argued against it, and I still think it was the wrong technical decision. One trouble with any pictographic system is that it is limited only by the number of things in the universe that can be discretely represented in an image. My name is John. To distinguish me from the other Johns, I have a second name, Hudson. To distinguish me from the other John Hudsons, I can refer to a number of facts about my life — I was born in England, I live in Canada, I design typefaces — as befits the circumstances. This nomenclature isn't efficient in the sense that needing only a single character code in a text message is efficient, but it is flexible and reliably allows me to communicate who I am even to people whom I have never met.
One trouble with any pictographic system is that it is limited only by the number of things in the universe that can be discretely represented in an image.
Now imagine if every individual on the planet sought to represent himself or herself in communications with a unique, personal pictogram? Well, the first thing that would be apparent would be that the encoding of emoji in Unicode is an insufficient and inappropriate technology. And I maintain that this was already true when the first emoji characters were proposed for encoding, because it is the virtually boundless nature of pictographic communication to expand. So Unicode encoded a set of what were essentially compatibility characters to support the superset of emoji images used by major cellular phone companies, and was immediately beset by requests for additional emoji characters, and for ways to modify the existing ones. Ironically, the methods devised to modify emoji characters, e.g. to represent diverse skin tones, immediately adopt the syntactic methods of language, of noun+adjective. Now people need to figure out how to 'say' things in the emerging syntax of emoji. I assume the process will be something like the way in which a pidgin becomes a creole by the instinctive imposition of grammar on an ad hoc, unstructured vocabulary. Conventions will develop — to convey narrative chronology, for instance, (do Arab emoji users arrange the images from right-to-left, I wonder? what confusions might arise there in cross-cultural communication?) — and unsurprisingly these will mimic the conventions and common structures of language. Because that's how we think.
If Unicode encoding is the wrong technical solution for emoji, what would be a better one? Ultimately, the expansiveness of pictographic systems demands the ability to send arbitrary images between devices in some fast and bandwidth-efficient way. The handling of emoji as text codes is, at best, a convenient temporary solution, free-riding the existing interchange mechanisms of text messaging. At worst, it is an avoidance of the problem that actually needs to be solved, which is how *not* to send emoji as text, but as something else.
Do Arab emoji users arrange the images from right-to-left,
I wonder? What confusions might arise there in cross-cultural communication?
Matt Gray and Tom Scott
Developers of the Emojili App, "the emoji-only network”
Are we headed toward a time when we communicate exclusively with images? Would this be considered progress or regression?
Nope. There's absolutely no evidence for that!
Emoji have their own subtexts based on the context they're used in, and shared interests and in-jokes. We've answered a few emoji-only interview questions: in each case, we didn't fully understand the question, and the questioner no idea what our response was.
When constructing something more than a couple of glyphs long, a lot of assumption can go into parsing it unless the exact emoji you need are there.
Also: you must take into account that emoji are rendered differently on different devices. Take(dancer) for example. Apple will render this as a female flamenco dancer while some versions of Android will show a male disco dancer.
Most vendors are using the Unicode Consortium's Emoji specifications, so at the moment it's up to the Consortium. Devices need to follow a standard otherwise an emoji from one device may not render on another!
Senior Fellow, Game Design & Digital Media College of Professional Studies Northeastern University
What I find most interesting about the Apple emoji expansion is how it throws down the gauntlet for other system manufacturers (Google, Microsoft) to also adopt this standard as quickly as possible. It's funny to have Apple be first out of the gate on something new, since usually they like to be "second, but best" (or as they prefer to say, they wait until they have the technology right). But in this case, it's a Unicode standard (more info here ) and Apple's just been the fastest to pick it up.
The tricky bit about giving emoji any race at all (including Caucasian) is that suddenly there's an opportunity to really see yourself in an emoji -- or to not see yourself, if you're not the proud owner of one of the six skin tones from the Fitzpatrick scale. What if you're a wavy-haired, freckled, pale olive complexioned Portuguese Scotsman? Do you pick the one curly-haired guy, or the freckled kid, or the second lightest tan skin tone? And what about facial hair? Of course at this point you're really getting into avatar design, which is something to which people are becoming more and more accustomed— whether it's customizing an animated video-game character or just uploading a photo of yourself to a chat board.
Of course, when choosing an avatar, one often carefully considers: Do I want the video-game character to look like me, or like who I'd like to be, or someone else entirely? For this dating web site's profile pic, do I want to upload my professional headshot photo that makes me look flawless but also kind of boring, or do I upload that embarrassing fell-asleep-on-the-beach picture that everyone loves that shows I'm funny, but might also imply that I'm lazy?
Emoji are a bit different. With an avatar, you might worry about making a first impression in virtually meeting someone, but emoji, at least at this time, are still understood as more of an "among friends" thing. My sister and I have our own "emoji language" of sorts that has grown from visual puns that we like to make. Last year, our mother got an iPhone and she agonized over picking which emoji was "her" -- the old woman (she rejected that one pretty quickly)? The pig (her favourite animal)? The smiley face with a halo (because she's so good)? She now changes emoji regularly, but [it is apparent that] my sister and I associate emoji with how we're feeling rather than who we are.
This is how emoji started to be used in the US. First we had emoticons like the smiley and the winkie, which helped us to interpret jokes and innuendo as we moved towards email and texting. Then we started importing the overseas versions that told little stories (like the ones that make up my standard signature -- throwing a table and putting it back). And then Apple put some of the emoji on the iPhone, and the world exploded.
And now Apple is leading the pack again in adopting the Unicode standard for diverse emoji. In a happy coincidence, I just updated my work computer to OS X 10.10.3 yesterday, so I'm able to actually see how they work. A good explanation (including about why the default skin tone is yellow, which will save me getting into it) can be found here.
What's cool about this, and what really gets back to answering your question about the future of emoji is this: The new skin tones are not just copies of the emoji in each of six skin tones but rather they are "character modifiers" -- much like hitting option plus a diacritical mark on your Mac and then pressing a key gets you an accented letter. For example, option + ` (backtick) followed by e gets you è (e accent grave). This is technically exciting because there is still plenty of unclaimed space in the Unicode character specification. Imagine if you could use a series of modifiers to specify "smiling man with olive skin, freckles, wavy hair, round face, goatee"? At that point we'd basically have a mini avatar generator on our keyboards!
Emoji are still understood as more of an "among friends" thing.
As for the tempura shrimp and the pile of poo, who can say what the future holds for them?
However, anyone who's ever tried to type a mathematical formula or a French phrase or a table-flipping emoticon on an iPhone knows that it can be a rather tedious process with all of the hidden keys. However, with its Watch, Apple may once again be pointing towards the future of emoji, since the little animated ball guy can be tweaked into many facial expressions (from what I've seen, anyway). In order to accomplish this, Apple had to go back to the generic yellow character (albeit with a bit of a Pac-Man 3D edge), but this is just version 1.0. Imagine a little "emoji dress up" app that lets the user flip through tons of features directly on the watch -- and maybe a phone or desktop app like Bitstrips that lets you tweak every detail and then sync to all of your devices via the cloud!
On the other hand, although I'm sure it's technically feasible, Apple might want us to stick to the generic yellow character so that we focus on the emotions being expressed rather than on what the character looks like. Certainly other features like the "share your heartbeat" feature point to abstractions that focus on emotional content versus visual form, but these things are by no means exclusive. Perhaps, in time, the Apple Watch will be able to read your "true" emotion and transmit the appropriate emoji along with your message without you even typing it in! That could be wonderful for the frequently misunderstood but awful for the passive-aggressive "just kidding" winkie-adders ("I hate it when you do that ;-)")
As for the tempura shrimp and the pile of poo, who can say what the future holds for them? In February 2015, the Dhyani Buddha emoji was removed from the Unicode 8.0 Draft spec, while the following "most popularly requested emoji" are still "on track for inclusion": Bottle With Popping Cork Burrito Cheese Wedge Hot Dog Popcorn Taco Turkey
Make of that what you will, but I think the future of the non-humanoid emoji mostly depends on who's most vocal in telling their (emoji) stories to the right people!