Tag Archives: augmented reality

Downfall

The Downfall of Humanity

Author: Shane Stone

“I fear the day that technology will surpass our human interaction. The world will have a generation of idiots”
Albert Einstein

Archaeologists like Nancy Tanner suggest that the creation of tools lead to bipedalism. They believe that for early humans to use tools they would need complete use of their hands; the only way to constantly have hand freedom is through bipedalism. This theory has even been supported through an anecdotal observation made by Jane Goodall, during which chimpanzees using tools moved by bipedalism (Tanner 1981). Since that time tools and technology have been a significant aspect of humanity’s culture. Mummies are found buried with ancient technology and people today do not leave their homes without their cell phones (Saenz 2013). Even in the time of Aristotle, he predicted that there would be further progressions in technology resulting in self-operating lutes and looms (Lanier 34). As technology continues to develop and become a more integral part of our society, humanity must consider a question: Will technology be the downfall of humanity? Our reliance on technology is apparent in how lost you may feel when you are without your cell phone or without service. With technology becoming more integral, humanity’s reliance grows, leaving this looming question. Today, most of society embraces technology and sees it as a promising horizon. Scientists foresee the use of robots and artificial intelligence in healthcare, industry, and service (“Domain-Specific Robotics Research” 2010). IBM’s Watson is capable of scanning and storing information, and then applying it to specific individuals and situations. Currently, Watson is being used in medicine and in November 2013, IBM announced that Watson would now be open to developers.

Picture sourced from: http://www.popsci.com/blog-network/zero-moment/cognitive-computing-all-ibm-releases-legion-watsons
Picture sourced from: http://www.popsci.com/blog-network/zero-moment/cognitive-computing-all-ibm-releases-legion-watsons

As this technology develops and introduced into society’s daily lives, the opportunity for Karel Čapek’s prediction to come true becomes more likely. In his 1921 play, Rossum’s Universal Robots, Capek foreshadows the potential fate of technology as “the product of the human brain [that] has escaped the control of human hands” (Čapek 1921). Despite this warning, humanity proceeds in its plans to develop and integrate technology into all aspects of life. Through this paper, the potential effects of technology will be explored by analyzing science fiction novels and movies and comparing these fictional worlds to today’s technological situation. Taking these fictional worlds into consideration, as well the opinions of academics in the fields of futurism, physics, and singularity, the effect of technology on the long term wellbeing of humanity will be determined.

The Threat of Technology from Novels

neuro
Image from http://www.select.art.br/article/da_hora/mente-conectada

In William Gibson’s Neuromancer the importance of technology on everyday life as well as the strength of artificial intelligence is realized. The reader is introduced to the protagonist, Case, who lives an apparently meaningless life. However, the reader learns that his life was different before, when he was able to “jack in” and enter into cyberspace. He is no longer able to “jack in” because a previous employer altered his body to prevent it. Because he was so dependent on his connection with technology, being cut off from it drove him into depression. Eventually, he undergoes a second procedure that allows him to reunite with the cyberspace he had grown so fond of. When he finally reenters “he [is] laughing…tears of release” (Gibson 52).  Additionally, Wintermute and Neuromancer, two artificial intelligence interfaces, demonstrate the power and influence of artificial intelligence. When General Corto was left in a catatonic state, no treatment proves successful until Wintermute “pull[s] him out of it” (95). Wintermute is able to control Corto for years until he attempts to regain control, at which point Wintermute kills him. When Case first discusses artificial intelligence with Molly he explains to her that most “Aren’t much smarter than dogs” (95) but Neuromancer acts as a counter this case when he reveals to Case how even his creator “couldn’t imagine what I’d be like” (269). Both artificial intelligence units demonstrate the potential power technology could have over man. Though Neuromancer is a work of fiction, some of the technology in the book is similar to some of the advances in today’s technology. For example, Molly is participating in what is known as Transhumanism, an idea and practice in today’s society that suggests that the “human race can evolve beyond its current physical and mental limitations by means of science and technology” (“What is Transhumanism?” 2013). People have already begun to make alterations to themselves like Molly, but they are referred to as Cyborgs, and have enhanced hearing and vision, metabolic enhancement, and brain computer interfaces (“Technology Powered Superhumans” 2013). Case is able to utilize Molly’s improvements to access her sensorium. In doing so he is able to see and feel what she does, but is unable to effectively communicate with her. Technology like this is not too distant from today either. The Nicolelis Lab at Duke University has suggested that the rubber hand illusion (RHI) can be developed in primates. The RHI suggests that subjects can “develop a sense of ownership of a mannequin hand when they view it being touched while tactile stimuli are simultaneously applied to their own hand” (Shokur et al 2013). Using this concept Shokur et al found that a monkey responded similarly to mannequin and actual manipulation. This parallels what Case felt with Molly. Their experience even inspired a game where two players work together to achieve a task in the physical world only using the information available to Case. One player performs the role of Molly and physically moves, while the player acting as Case has the information needed to communicate by alternating a screen between red and green. Combining these technologies would effectively recreate what occurs in Neuromancer. As some technological aspects of Neuromancer come to fruition, it makes sense that other technology from the book, like artificial intelligence, may also come to fruition.

Rick Moss creates a reality similar to our own in the Ebocloud where humanity has passed Facebook and is now entranced by Ebocloud, an online community where people have been assigned to Ebos or tribes based on a detailed questionnaire applicants fill out to become members. As the system grows it becomes enrooted in society and Ebos become an important part of off-line life. Eventually a new technology is introduced that connects Ebo members’ brains to the cloud. It allows them to know when Ebo members are nearby and to sync themselves to applications on digital devices. Throughout the novel the reader is left to believe all of the advancements are just to enhance the system, but upon the book’s conclusion Radu, Ebocloud’s creator, explains what he is trying to achieve. Over the next few decades, he plans to introduce new technologies to ease them into it and to eventually “[take] humans forward along their evolutionary pathway” (Moss 434). According to Radu, this evolutionary pathway is driven by humanity’s drive to be a part of a meta-system; the connections established by the tattoos are just the first step of development (434). In the epilogue the reader is privy to how the program has developed further. El, the novel’s protagonist, discusses with Jared about how he “came back off the cloud” which suggests that the cloud has

Image from: http://www.barnesandnoble.com/w/ebocloud-rick-moss/1100075558
Image from: http://www.barnesandnoble.com/w/ebocloud-rick-moss/1100075558

become similar to the matrix of Neuromancer in that users can enter and exit this virtual world. Though such technology seems farfetched the preliminary steps to creating it exist today. In the Nicolelis Lab, they have created the first brain to brain interface that allows for semi motor information to be shared between two rat brains, which is analogous to members of the same Ebo sensing each other. Additionally, tattoos are already being created to connect humans to technology and allow control over them. In fact, Google has already applied for a patent on one such technology that would utilize sub vocalizations to allow wearer to communicate silently and wirelessly. Chris Grayson, an augmented reality expert, predicts that in the future first responders will be able to make sense out of what is happening upon arrival at an incident by using some form of map overlay. Using the digital tattoos, an app called “Helping Hand” is developed in Ebocloud to allow for just that response.  In terms of the life presented in Ebocloud, Moss is not the first writer to consider applying ebos to American society. In a 1997 interview, fiction writer Kurt Vonnegut suggested that expanding the family unit to create extended families would resolve problems in American culture, including divorce rate. However, not all academics are like minded. Aristotle believed that humans interacted with other humans out of necessity not desire, and would rather be alone (Rackham 1932). Jaron Lanier, an American writer and computer scientist, defends Aristotle’s position citing how Roman generals were reward after years of fighting with land so that the remainder of their days could be spent left alone. Lanier even cites the American government’s utilizations of this same technique by offering settlers the same prospect if they moved out West (Lanier 35). If people desire to interact with others such a reward would not be so enticing.

Rainbow’s End, a Vernor Vinge novel, also presents a world that appears to be not so distant from our own. Citizens wear contact lens that allows the wearer to interact with the digital world and augment reality with visual layers. More simply, they are a more developed and advanced Google Glass. Rather than verbally commanding it, wearers are able to mentally control what their lens achieve. This novel perhaps best suggests how society could proceed because the technology is so similar to modern devices; in this world users can message each other silently on the lens just as users of Google Glass can today. In the novel, with the advances in technology, some adults, even a Nobel Prize winner, are considered worthless because of their inability to successfully utilize the new lens technology. This is mirrored by today’s world where nearly half of all jobs require computer literacy, and that number is expected to increase. Currently, studies suggest that 82% of adults are computer literate and only 54% of seniors are capable. Considering this, the job market becomes more challenging for adults and will only get harder as the information age progresses. However, the most troubling aspect of the Rainbow’s End reality is that code can reproduce. Though this may not seem like a problem, French Archeologist André Leroi-Gourhan believes that “once Homosapiens had equipped such machines with the mechanical ability to reproduce themselves, there would be nothing left for the human to do but withdraw into the paleontological twilight” (Leroi-Gourhan 248). If this is the case then humanity is in trouble because according to a NASA study, robot self-replication is a matter of engineering, and really does not require more technology than is already available today. NASA is hopeful this technology will lead to colonization in space, but this is not necessarily as good for humanity as they may anticipate.

The Threat of Technology from Movies

Beyond these literary works, there are films that focus on the theme of technology being the downfall of humanity. For example the 2004 film I, Robot takes place in 2035 where robots act as servants to humanity. These robots are based upon the 3 Laws of Robotics that Isaac Asimov introduced in his 1942 short story “Runaround.” According to these laws:

1)   “A Robot must never harm a human being or, through inaction, allow any harm to come to a human

2)   A robot must obey the orders given to them by human beings, except where such orders violate the First Law

3)   A robot must protect its own existence unless this violates the First or Second Laws” (Asimov 1982)

With these laws in place, it is believed that humans and robots will be able to seamlessly coexist. However, when an artificial intelligence unit for US Robots (a robot manufacturer) interprets the laws in a new way, things go wrong. The unit, called VIKI, realizes that humans are too self-destructive, and that for humanity as a whole to be saved some people must be killed. Fortunately, in this film humanity gets lucky because Will Smith’s character with the help of a robot figures it out, and stops her. Nonetheless, if a similar situation were to occur in today’s society would someone realize in time to stop the robots?

In other films like WALL-E and The Matrix no one is able to stop the robots initially, and it takes a couple centuries for humans to take back control. Of the movies discussed The Matrix seems the most like a warning to humanity because it “takes place” when the movie was released, in 1999. In actuality the film takes place in 2699, but most of humanity is trapped in the late 20th century. It is more from the prequels, housed in the short film collection The Animatrix that insight is given into how humanity in The Matrix film gave way to robots. According to the Matrix’s history, told in the short film The Second Renaissance, at the end of the 20th century humanity celebrated the creation of

Image from: http://www.comicvine.com/forums/battles-7/kilg-re-vs-neo-matrix-1494633/
Image from: http://www.comicvine.com/forums/battles-7/kilg-re-vs-neo-matrix-1494633/

artificial intelligence and robots. Then, a robot named B166-ER killed its owner which resulted in a robot rights movement. In the end, all robots were ordered to be decommissioned. Rather than be destroyed, they left to establish their own nation called nation 01. As a result of robot efficiency they dominated the economy and became a threat to humanity. In response, the human nations attempted to attack nation 01 with nuclear weapons but the robots were not as affected as anticipated. After being threatened the robots went on the offensive and after much success attempted to negotiate with the United Nations, and were rejected. As a last resort, the nations joined together to block the sun, thereby cutting off the robots energy source. Although the humans were briefly rewarded with this tactic, the robots responded by taking advantage of the energy created by the human body. When the robots ultimately won they turned humans into a crop and harvested energy from humans while keeping their minds occupied in the Matrix. Despite the machine success, the theme of one person changing the world and restoring balance is seen as Neo, the protagonist, ends the 600 years reign of machines by fostering a truce between the sides and making peace. Though these ideas seemed more like fiction than a prediction in 1999, today’s technology could lead to the world similar to The Matrix. If NASA proceeds with its robot space colonization, NASA would be facilitating in the formation of nation 01. If the robots have the ability to reproduce like NASA has suggested then they could develop and challenge Earth’s economy as well. Scientists are also facilitating the robot’s ability to harvest energy from humans in some of today’s energy research. In many modern medical devices, lithium batteries are considered the most effective power source, but scientist are looking for forms of renewable energy. A recent paper has found Glucose Biofuel Cells to be the most promising of these options (Zebda et al 2013), but another group has suggested utilizing the heat humans naturally produce (Dvorsky 2013). The latter, would eerily echo the concepts used by the machines in The Matrix.

WALL-E (2008) takes place in a future where humanity’s disregard for nature resulted in Earth becoming uninhabitable. As a result, in 2105 Buy n’ Large, a corporation, moves humanity from the planet to a ship where they will live until Earth is again habitable. Unfortunately, the viewer learns that in 2110, the ship’s artificial intelligence unit is given a no return directive because Buy n’ Large has given up hope for Earth. The movie, takes place 700 years later in 2805, when humans have lost touch with Earthly lifestyles,

Image from: http://earnthis.net/brian-terrills-100-film-favorites-30-wall-e/
Image from: http://earnthis.net/brian-terrills-100-film-favorites-30-wall-e/

spending their lives in chairs and relying upon robots and automated features of the ship. Christian Journalist, Rod Dreher, explains how in this scenario the viewer sees how their reliance on technology has resulted in the “[loss of] what makes them human” (Dreher 2008). The director, Andrew Stanton, expanded upon this point explaining how the robots eliminated humanity’s need to establish and put effort into relationships. Since the humans aboard the ship’s lives revolved around technology rather than each other they had no need to do this. Luckily, as in I-Robot and The Matrix, a special robot, WALL-E, changes everything and helps to restore balance.

Even in works of science fiction where humanity’s downfall is not a major theme, the technology present in them is not as out of reach as it once seemed. Minority Report is a 2054 science fiction film, based on a short story by Philip K Dick, and a lot of the technology presented in the film is being developed or used today. In the film, a citizen’s biometric data allows corporations to access their government profile so that marketing can be targeted more accurately.

Google is now developing the technology that will utilize environmental sensors so that advertisements on electronic devices will be “targeted ads tailored to fit with what you’re seeing and hearing in the real world” (Whitehead 2012). Also in the film, psychic humans, known as ‘precogs’, are used to predict crime before it happens. Although those humans have yet to be identified today, Viktor Mayer-Schonberger and Kenneth Cukier suggest that big data can be used in such a way so as to make this ‘precrime’ effectively possible. In fact, trends with ethnicity, gender, breathing, and heart rate to predict crimes have been applied in the field, but more work is needed before widespread application occurs. Even one of the most iconic science fiction movies, Star Wars, has inspired an idea among defense systems. Philip Lubin of the University of California Santa Barbara and Gary Hughes of California Polytechnic State University believe that a real life death star could be constructed to defend earth from an oncoming asteroid threats. Since technology from science fiction films that aids humanity has become more available, the technology that destroys humanity must not be too far behind.

The Future

Companies have even realized the potential of science fiction in their development and have reached out to futurists and science fiction writers to work for them. One such example is futurist writer, Alvin Toffler, who has helped IBM, Xerox, and AT&T to facilitate the organizational impacts of computers and to offer strategic advice. Additionally, Intel’s Tomorrow Project explores the world’s possible futures by engaging in discussion based on science-based fiction because they recognize that science fiction can have dramatic changes on the technology of the future. The Creative Science Foundation even holds an annual conference that allows for scientists, designers, and artists from around the globe to discuss and explore the consequences and uses of future technology on society. Finally, and perhaps the most notable, was Google’s hiring of futurist Ray Kurzweil as director of engineering in 2012. This hire not only created buzz for Google, but introduced an expert on machine learning, reading, and talking that can help in their development of future products.

Technology corporations utilizing science fiction in their product development is just another example of the blurred lines between fiction and reality. As this transition continues, the real question is whether or not academics believe the fate of humans in many science fiction films is the fate of today’s society. According to Moore’s Law, the computing power of technology will double every two years (Moore 1965). Extrapolating Moore’s Law, Kurzweil predicts that computing technology will be equivalent to human technology in 2020 and by 2030 Hans Moravec predicts that robots will be a prominent part of society.

Considering that Moore’s Law has been accurate since its creation 50 years ago, and even holds true when you include technology development since the hand cranked computer (therefore 100 total years), Kurzweil’s prediction seems accurate. At this point, Cambridge University philosophy professors believe that computers could become cleverer than humans. As this proceeds it could lead to a scenario similar to what occurs in science fiction movies. According to Lanier’s opinion that “technological culture influences what technologists create” would likely result in a society like WALL-E (Lanier 121). In today’s society obesity has become a bigger health crisis than hunger. With today’s culture being lazier, it would suggest that technology will develop to accommodate this, and slowly but surely would lead to what is seen in WALL-E. In order to visualize the potential progressions of humanity I have created a flowchart. These progressions are based on what happens in WALL-E, I,Robot, and The Matrix, so it is not every possible scenario, but one a few. If this is the case then humanity should adhere to Murphy’s Law and recognize that would could go wrong will go wrong, and unless someone comes along to save humanity (like in I, Robot, The Matrix, and WALL-E), should prepare for the worst.

Flow Chart made with bubble.us
Flow Chart made with bubble.us
Interact by following this link: https://bubbl.us/?h=1c647e/391a34/18PZPh7OTTkhA

In actuality it appears as if humanity is actually not in danger. According to Gordon Moore, the creator of Moore’s Law, the exponential development “can’t continue forever” and academics like Kurzweil are applying his law incorrectly. Moore intended for his rule to apply to semiconductor circuits, not all technology. The reason why it cannot continue forever though is physics. Currently silicon is the best material for creating circuits, and according to theoretical physicist Michio Kaku, the transistors will eventually generate so much heat that the chip will melt and electrons will be lost.

However what may be the most convincing is the argument made by Singularity Hub writer Aaron Saenz. He explains how the robots that humanity has come to fear are no more than Hollywood using robots as symbolism of human worries. Terminator represents the communist fears of the 1980s, the Decepticons represent the energy crisis, and The Matrix machines represent the fear of global warming. Although some of the threats are propaganda, other threats, such as a large robot workforce, continue to frighten human workers. Since 2008, the amount of automated robots being used in the industry has increased nearly 50% and recent studies have suggested that nearly 50% of today’s jobs could be achieved by machines. According to Edward Leamer, director the UCLA Anderson School of Management, “if you have nothing to offer the job market that cannot be supplied better and cheaper by machines… plan on doing low wage service.” Even low wage service is threatened because companies like Foxconn, that already offer low wage service, are starting to utilize more robots. As more robots enter the work force, the threat of unemployment appears to increase. However, threats of a robot run economy and false ideas created by Hollywood are nullified by Saenz’s main point, “we own technology” (Saenz 2013). As long as we are aware of the capabilities of technology and maintain the correct safeguards, humanity has a better chance of being destroyed by a nuclear winter than by technology.

Works Consulted

Animatrix. Dir. Mahiro Maeda. Warner Home Video, 2003.

Aristotle, and H. Rackham. The Politics,. London: W. Heinemann, 1932. Print.

Asimov, Isaac. The Complete Robot. Garden City, NY: Doubleday, 1982. Print.

Barrie, Allison. “Massive, Orbital Laser Blaster Could Defend against Asteroid Threats.” Fox News. FOX News Network, 19 Feb. 2013. Web. <http://www.foxnews.com/tech/2013/02/19/massive-orbital-laser-blaster-could-defend-against-asteroid-threats/>.

Choi, Charles Q. “Temporary Tattoos Could Make Electronic Telepathy and Telekinesis Possible.” Io9. N.p., 20 Feb. 2013. Web. <http://io9.com/5985558/temporary-tattoos-could-make-electronic-telepathy-and-telekinesis-possible>.

Dignan, Larry. “Google Hires Kurzweil: A Look at the Returns.” ZDNet. Between the Lines, 15 Dec. 2012. Web. <http://www.zdnet.com/google-hires-kurzweil-a-look-at-the-returns-7000008844/>.

“Domain-Specific Robotics Research.” IFTF: Robot Renaissance Domain Papers. Institute for the Future, 2010. Web. <http://www.iftf.org/our-work/people-technology/technology-horizons/robot-renaissance/robot-renaissance-domain-papers/>.

Dreher, Rob. “Wall-E.” Beliefnet. July 2008.

Dubash, Manek. “Moore’s Law Is Dead, Says Gordon Moore.” TechWorld Rss. N.p., 13 Apr. 2005. Web. <http://news.techworld.com/operating-systems/3477/moores-law-is-dead-says-gordon-moore/>.

Dvorsky, George. “A Chip That Turns Your Body into a Battery.” Io9.com. Io9, 15 Jan. 2013. Web. <http://io9.com/5976148/a-chip-that-turns-your-body-into-a-battery?tag=futurism>.

Garcia, Cecilia. “New Digital-Divide Campaign Would Leave Seniors Behind.” Growing Older, Getting Poorer. New America Media, 25 Mar. 2013. Web. <http://newamericamedia.org/2013/03/new-digital-divide-campaign-would-leave-seniors-behind.php>.

Gibson, William. Neuromancer. New York, NY: Ace, 2004. Print.

Gross, Doug. “Google Patenting an Electronic ‘throat Tattoo'” CNN. Cable News Network, 13 Nov. 2013. Web. <http://www.cnn.com/2013/11/12/tech/innovation/google-throat-tattoo/index.html>.

How to Stop Robots From Killing Us. Perf. Michio Kaku. Youtube. The Big Think, 31 May 2011. Web. <http://www.youtube.com/watch?v=JPVOPzYiCeg>.

Hui, Sylvia. “Cambridge to Study Technology’s Risk to Humans.” NBC News. Associated Press, 25 Nov. 2012. Web. <http://www.nbcnews.com/technology/cambridge-study-technologys-risk-humans-1C7206889>.

I, Robot. Dir. Alex Proyas. Perf. Will Smith. Davis Entertainment, 2004. Film.

Lanier, Jaron. Who Owns the Future? New York: Simon & Schuster, 2013. Print.

Leroi-Gourhan, André. Gesture and Speech. Cambridge, MA: MIT, 1993. Print.

The Matrix. Prod. Andy Wachowski and Larry Wachowski. Dir. Andy Wachowski and Larry Wachowski. By Andy Wachowski and Larry Wachowski. Perf. Keanu Reeves. Village Roadshow Pictures, 1999.

Moore, G.E. “Cramming More Components Onto Integrated Circuits.” Proceedings of the IEEE 86.1 (1965): 82-85. Print.

Moss, Rick. Ebocloud. New Orleans: Aqueous, 2013. Print.

Olympian. “State Libraries, Microsoft Will Hone Computer Skills.” The Olympian [Olympia] 15 Nov. 2013, Opinion sec.: n. pag. Print.

Pais-Vieira, M., Lebedev, M., Kunicki, C., Wang, J. & Nicolelis, M.A.L. A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information. Sci. Rep. 3, 1319; DOI:10.1038/srep01319 (2013).

Robo-everything, 2008 DVD featurette, Walt Disney Studios Home Entertainment.

Rothman, Peter. “Technology Powered Superhumans.” H Magazine. N.p., 21 Nov. 2013. Web. <http://hplusmagazine.com/2013/11/21/technology-powered-superhumans/>.

Rothman, Peter. “What Is Transhumanism?” H+ Magazine, 22 Nov. 2013. Web. <http://hplusmagazine.com/2013/11/22/what-is-transhumanism/>.

Saenz, Aaron. “Welcoming Your New Robot Overlords.” Singularity Hub. N.p., 16 July 2013`. Web. <http://singularityhub.com/2013/07/16/welcoming-your-new-robot-overlords/>.

Sofge, Erik. “Cognitive Computing For All: IBM Releases a Legion of Watsons.” Zero Movement. Popular Science, 14 Nov. 2013. Web. <http://www.popsci.com/blog-network/zero-moment/cognitive-computing-all-ibm-releases-legion-watsons>.

Tanner, Nancy Makepeace. On Becoming Human. Cambridge: Cambridge UP, 1981. Print.

Toffler, Alvin. “The Partnership.” Biography of Alvin+Heidi Toffler. Alvin+Heidi Toffler- Futurists, 2013. Web. <http://www.alvintoffler.net/?fa=biospartnership>.

“The Tomorrow Project.” Intel, n.d. Web. <http://www.intel.com/content/www/us/en/research/tomorrow-project/the-tomorrow-project.html>.

Vinge, Vernor. Rainbows End. New York: Tor, 2006. Print.

Vonnegut, Kurt, Jr. “Interview with Kurt Vonnegut, Jr., Sept., 1997.” Interview by Clifford L. Meth. Interviews. Clifford Meth, 2004. Web. <http://www.cliffordmeth.com/methonvonnegut.htm>.

WALL-E. Dir. Andrew Stanton. By Andrew Stanton, Jim Reardon, Ben Burtt, Elissa Knight, and Jeff Garlin. Prod. Jim Morris. Walt Disney Studios Motion Pictures, 2008.

Whitehead, John W. “Minority Report: Ten Years Later, Fiction Has Become Reality.” The Huffington Post. TheHuffingtonPost.com, 17 Sept. 2012. Web. <http://www.huffingtonpost.com/john-w-whitehead/technology-surveillance_b_1854117.html>.

Zebda, A. et al. Single Glucose Biofuel Cells Implanted in Rats Power Electronic Devices. Sci. Rep. 3, 1516; DOI:10.1038/srep01516 (2013).

Thank you to Amanda Gould, my classmates, friends and the staff at the Duke University Writing Studio for helping me to complete my final assignment.

Science DataVis

Utilizing Media Tools to Augment Scientific Journal Articles

Author: Kim Arena

1. Introduction to Scientific Data

Scholarly journal articles are the primary means through which scientists communicate their research to the scientific community, and through reading and sharing these articles they are exchanging data. How this data is interpreted and received depends on how it is understood by the audience. For other scientists who are familiar with the material, the articles can be conceptualized and the data becomes relevant. But how much information does the data truly hold?

The way that we visually view text alters the way we understand its content, thus augmenting text expands our interpretations and increases the limit of knowledge that can be obtained with each form it takes. In her novel How We Think: Digital Media and Contemporary Technogenesis, N. Katherine Hayles explains, “The more one works with digital technologies, the more one comes to appreciate the capacity of networked and programmable machinery to carry out sophisticated cognitive tasks, and the more the keyboard comes to seem an extension of one’s thoughts…” (3, 2012). Augmenting the text of scientific literature can alter the reader’s perception and interpretation of both the literature and the data, establishing a connection between the concrete and abstract ideas associated with scientific research and expanding the range of understanding that can be obtained from the work. By receiving the information in a one directional manner, we limit the range through which the data can be received, and subsequently limit the interpretations and influences the data holds. What if we expanded the audience that could understand the data? People think differently when data is presented through different medias, therefore why not expand the boundaries of how data is represented? Remediating scientific journal articles through a variety of text and data analysis tools can clarify data and expose patterns that may have been overlooked. Remediation, a concept created and explored by Jay David Bolter and Richard Grusin, argues that new forms of visual media enhance and rival traditional media forms, thus establishing their own cultural significance. In this case, the added dimension of the media elements establish a deeper meaning that goes beyond the confines of text. In this essay, I will expand Bolter and Grusin’s argument and explore how remediations of scientific works through various visualization tools become meaningful in relation to the data they present.

2. Remediation of Data

Different forms of representation make certain elements of scientific research easier to understand for certain people. If people think in different ways, why not mediate data in different ways too? For scientific data, interpretation is the key to extracting the most information and subsequently utilizing that information to make advances in research. Scientists are constantly exchanging data by reading others articles and basing their own studies off of the information they gather. Remediated data presents an alternative way of “reading” scientific literature that can expand the areas of relevance and importance of the data itself. It can highlight aspects that may not be presented in the scientific literature, but that are related to the impact the goal of the research will have. Cultural theorist Lev Manovich defines information visualization as “…a mapping between discrete data and a visual representation”, and distinguishes a difference between information and scientific visualization, claiming “…information visualization is distinct from scientific visualization in that the latter uses numerical data while the former uses non-numeric data such as text and networks of relations” (Manovich, 2011). Information visualization allows us to explore what lies behind the numbers, as the text of an article is just as much data as the numerical information is.

3. Utilizing Text Analysis Tools

Using certain text analysis tools, such as Wordle, can establish a relationship between words used in the article by highlighting the ones most commonly utilized. Wordle is a

Entering text, such as the data from this essay, into text visualizers like Wordle augment the way that the data is perceived by the reader.
Entering text, such as the data from this essay, into text visualizers like Wordle augment the way that the data is perceived by the reader.

tool that generates an image based on the reoccurrence of words throughout a text, in this case scientific articles. The image is randomized, though sizes of words are relevant to the frequency at which they occur, highlighting their importance. The generator allows the creator to manipulate the general orientation of the words and the colors that are incorporated as well. Overall, the piece becomes an interpretive form of an in depth work. This presentation of information demonstrates important elements of the research, extracting the main ideas represented by the specifics of the data. This visualization strips the data of the complex aspects and simplifies it in a way that can be understood by a multitude of audiences, regardless of their level of scientific knowledge. This visualization alone would not necessarily have a major impact on research, and should not be used as a source of information on its own, yet when paired with the scientific article itself, it introduces artistic elements to a concrete idea that can help to increase understanding of the article and its relevance in relation to applications. Manovich explains, “Displaying the actual visual media…helps the researcher to understand meaning and/or cause behind the pattern she may observe, as well as discover additional patterns” (Manovich, 2011).

Scientific literature can be confusing and can contain many terms that are not understood by people unfamiliar with the particular field of research. Visual representations, like Wordle, text2image, and various Voyant Tools, can allow any person to be able to appreciate the importance behind scientific research and understand the relative connections and interactions between particular elements; the art offers a different perspective, and the way a piece is perceived determines how it will be determined and thus how its’ significance will be decided. The breakdown and remediation of research allows the reader to grasp a basic understanding of the subject, and because it is presented in an unconventional manner, it expands our interpretation of the subject. The article provides the data; the visualization kick starts our brain to think in alternative manners. In Wordle, for example, the interpretations are not influenced only by the words, but also by the orientation of the words, the colors of the words, their sizes, and their position relative to one another. The artistic elements connect with the reader’s emotions and thus gain access to past experiences, therefore every person will experience the visualization in a different way. It could remind them of a family member, of a specific moment; the generation of words could simply set a certain mood. In this case, these different effects come together to shape perception and thus attitude towards the research that lies behind the art.

4. Data Visualization

The multitude of interpretations that can be extracted from such a simple concept as a word cloud open doors to the utilization of scientific research and expands its reach, both in relation to audiences and information. On a simple level, the visualization of research increases the amount of people who can become informed of research that is being conducted. This can especially be helpful for people who are interested in learning about a particular disease, or similarly for people who have been personally affected by a certain illness. The most accurate way to find information regarding research would be through direct scientific journals, yet the figures and jargon used in the articles is often over the average person’s head. The textual visualization can also be used to spread awareness about a disease and the research that is being conducted. Take breast cancer, for instance. Many people are aware of what it is and that it affects an increasing number of people every year, but how much do they really know about the science behind it? Visualizing a scientific article on the affect a particular gene has on the disease introduces terms such as “inhibiting”, “suppression”, “agents”, “oglionucleotides”, “overexpression” and others throughout a spatial design. When the textual visualization is first seen, the two words that stand out the most are “breast” and “cancer”, surrounded by the specific terms that make up both this particular image and the disease itself. The image is a piece of art, composed of research. The introduction of new words, presented in association to

Inserting scientific journal articles into text visualizers, such as Wordle, establishes an artistic representation of the data that expands the range of interpretation.
Inserting scientific journal articles into text visualizers, such as Wordle, establishes an artistic representation of the data that expands the range of interpretation.

known words, can prompt viewers to delve further into the science aspect. The artistic aspect can also open ideas regarding the research that the concrete text itself cannot, and though these ideas may not directly affect future research, it expands the influence of the research conducted and leads to an overall increase in knowledge. The added dimension also opens the opportunity for connections to be made. Everything thinks differently, and sometimes simplifying things could cause the viewer to visualize associations that may be shielded by the complexities of a conventional research journal article. The artistic element itself should not be overlooked as well. The colors and orientations of the words create an atmosphere that goes beyond the hard facts. For instance, a black background behind pink words can associate the dark world surrounding breast cancer being brightened by the research conducted and updated everyday.

However, textual visualizations, such as Wordle, should not replace scientific articles but rather can be used alongside them as an additional element. Wordle cannot relay facts and would not allow for any conclusions to be made in respect to the data that was collected. The use of Wordle as a visual representation is merely that–a representation, one that cannot stand on its own as an information source. Using Wordle as the primary form of interpreting data would lead to confusion and misinterpretation, as the randomization factor of the tool could cause certain aspects of the orientation of words to become misleading in their meaning. Worlde can be used to spark interest in a topic, but it cannot itself become a source of data. That being said, it does not mean that Worlde is entirely meaningless as a visualization, as its uses were previously described. When experiencing the word cloud, it must be taken into consideration that it is not a source of data, but rather an artistic interpretation. The reader must understand that Wordle does not communicate information but rather augments it. This augmentation of data can communicate themes and topics, but not concrete data. Misunderstanding this distinction could take displace the reader even farther from the true data, thus it is important to understand the role that the media visualization plays in relation to the data it involves.

5. Data as Information

Scientific articles can also be enhanced in ways that augment text to create interactive elements, thus putting the reader directly in contact with the information. The tools used to generate such elements allow the viewer to explore the article on their own, adding a dimension of discovery that complements the purpose of the research. Interactive representations incorporate scientific and artistic components while also adding a learning aspect that forces the viewer to work together with the information to establish an understanding. The level of understanding that accompanies the tool is up to the viewer and how they interpret the data as they explore the media. The context of the article can be comprehended in relation to how it is expressed by the media tool. For

Entering scientific journal article text into interactive tools, such as Word Count Fountain, allow for the reader to interact with the text and increases the cognitive depth of the data.
Entering scientific journal article text into interactive tools, such as Word Count Fountain, allow for the reader to interact with the text and increases the cognitive depth of the data.

example, Voyant Tools’ Word Count Fountain reveals a text based on word relevance, similar to Wordle. However, in this form each word is represented by a bouncing dot that travels across the screen, forming a stream of dots. The amplitude of the dot corresponds to its relevance in the article, and the viewer accesses the word by mousing over the dot. The constant movement of the dots causes the words to flash on the screen as they overlap with each other, subconsciously linking the words to one another as they are read off the screen. Moving the mouse around gives rise to more words and numbers, thus the viewer can piece together the article, as the more and more words are uncovered. The differing interactions that vary between each viewer allow for a multitude of interpretations of the data based on how the words are revealed and how the brain connects the interactions that are experienced.

6. Data as Art

It is not just the words in a research article that are important—after all, words themselves are only representations of concepts that we have come to accept. The actual data, not just the textual descriptions, is also important when interpreting a scientific article. The data, too, can be manipulated and viewed in an artistic and visual manner using a variety of media tools. Data isn’t restricted by the confines of text and numbers and can be organized into visual components. Conventionally, graphs and figures are generated and illustrated to demonstrate relationships between data, a form of visualization that allows us to establish connections between particular components. While these forms have artistic elements in and of themselves, the audience to which they speak is restricted, as the interpreter must be able to understand the visual and that often requires background knowledge on the subject matter to fully grasp the connections. Visual media can take data and transform it into a visual that can be interpreted in a variety of different ways, and no specific way is necessarily correct. That is the beauty of art and danger of visualization—it is open to be perceived through the eyes of the viewer as well as the eyes of the artist.

One media tool, text2image, demonstrates this concept of turning data into art, and thus opening its interpretations. While a graph or figure has a set interpretation that it aims to portray, art takes data and presents it in a different light that may not have a directive. Rather, the piece adds a dimension that aims to simply broaden the ways the data can be understood. This could be good or bad, depending on how the data is interpreted. Much like Wordle, misinterpretations could deviate the reader from the factual aspects of the data. Text2image takes data and converts it to an image that varies in pattern and color. While the image may not aim to extract a specific point from the data, the image does not

Inserting the protein sequence for Brca1, a protein that plays a major role in breast cancer research, into the text2image generator creates an artistic image that can be interpreted alongside the scientific information of the protein.
Inserting the protein sequence for Brca1, a protein that plays a major role in breast cancer research, into the text2image generator creates an artistic image that can be interpreted alongside the scientific information of the protein.

strip it of all meaning. For example, using text2image, the sequence of Brca1, a gene that plays a major role in breast cancer and is a main focus in cancer research, can be input into the generator and an image can be created, thus transforming the gene into art. The art doesn’t necessarily contribute to the research, but it shines a new light on the research that is being done. Linking art to science brings out the beauty of the components that make up our world—even if those components sometimes shed a negative light on our lives. The artistic elements can represent aspects of the research beyond the data itself—such as the successes and failures the researchers have encountered. The piece generated using the Brca1 gene sequence created two images: a black and white one with jagged lines and one with yellow and green faded patterns. The interpretations of the patterns are limitless

Using the same data and same tool can produce different images, thus the range of interpretations that can be generated from utilizing these tools is expanded even further.
Using the same data and same tool can produce different images, thus the range of interpretations that can be generated from utilizing these tools is expanded even further.

and the meaning behind the art is debatable. The black and white image could be viewed as the challenges faced when trying to understand the complex interactions of the gene and unlocking its role in cancer, while the yellow and green image could be interpreted as the hope in finding a cure that continues to grow as scientists uncover more and more about the gene and its functions, though the black line at the top could represent that there is still pieces that we have yet to discover—the image in not complete. This interpretation could be completely different from another person’s, and one could completely disagree with the interpretation. The point of the art, however, is that it established the means for these kinds of interpretations and stimulates our brains into thinking about the data in this unconventional form. Sharing ideas and perceptions increases the range of what we can learn from data and stimulates intellectual conversations that links scientists, research, and the average person.

These are only a few examples of tools that can be used to generate visualizations of data, yet they demonstrate the main concept that art can be used to represent data in a way that increases the range of understanding that can be subsequently extracted from it. Further, compiling all these data forms into one space, such as a tumblr, allows the reader to experience the images in relation to each other as well as in relation to the scientific articles. A tumblr compilation creates a portfolio of the images—and thus the data—much like how a journal compiles articles of relevance. In addition to reading a journal, one can view the tumblr page and access the data in an augmented manner. Captions and links to the original article accompany the images and visualizations, giving the viewer some background information and hinting towards the artist’s intention in generating the image while leaving the majority of the interpretation up to the viewer. This guidance establishes a basic connection between the data and the art without sacrificing the unlimited perceptions that accompany the visuals. Through a tumblr, viewers can additionally add comments online, thus stimulating conversations about their evaluations of the pieces that could affect the way others experience the medias. This sharing of information and thoughts is derived from the data itself, yet stimulated by the media, thus establishing an alternative connection between science, media, and theory.

7. Neurological Impacts of Pairing Literature with Media

Transforming scientific articles into media visualizations using various tools, such as the ones previously explained and arranged in the tumblr gallery, augments the literature of science. This form of E-literature, generating and augmenting texts through media tools, can change the way a story is experienced and thus increase the understanding that can be obtained from it. The altered presentations of the text allow for us to think in different ways, and such thinking opens doors for new neural connections to be made, thus expanding what can be taken away from the words themselves.

Much like how a graphic novel uses illustrations in place of words to expand meaning and affect the reading experience, visualizations give words a new meaning and that allows us to go beyond the textual context and delve into how the information links to our lives, to other information, and to other elements. For example, Young-Hae Chang Heavy Industries explores the interactions between music and literature, similar to how we have been exploring interactions between visualizations and scientific literature.  Young-Hae Chang Heavy Industries  pair their jazz pieces with a movie-like stream of words and phrases of varying sizes, speeds, and colors. Though the words do not form complete sentences, they are associated in a way that they flow with the music; rhythmically creating a disjointed stream of consciousness that comes together to tell a story. It is together the display, music, and the words themselves that are telling the story, and thus how the story is unfolds depends on the interpretation of the receiver, and how they associate the visual, sonic, and textual components. Thus, the result will be slightly different for everyone, and may even be different for a single person each time they listen. Similar to Wordle word cloud arrangements, the patterns, colors, sizes, timing, and orientation of how the words appear on the screen in addition to the blinking and changing backgrounds augment the way that the meaning behind the text is associated along with the changing beats and notes of the music. These two artistic components are intertwined to establish an ultimate experience that changes that way we interpret both, interplaying off each other to affect the way each is perceived in relation to the other. Words can invoke feeling just as music can, and that feeling can affect how you interpret the music, or vice versa. This specific kind of experience differs for each person—different things stand out to different people and thus have different significance to different people. Subsequently, each person will have a different interpretation of the piece, and thus the story. These components can even establish a mood, and thus they are not only augmenting the perception of the work itself, but also the environment through which the receiver extracts such interpretations.

Young-Hae Chang Heavy Industries project adds an artistic element to written work that serves to enhance it and provide a mean through which we can expand the boundaries of both music and literature to heighten the potential of both mediums and increase our levels of interaction and interpretation. The music could stand on its own, yet its full impact is not reached until it is paired with the literature; scientific articles can be read on their own, yet adding interactive media expands the reach of data in ways that it can interact with and by perceived by our minds. Literature should not be bound by the confines of print, but rather should reach out to other mediums to build on what the print has given us; to work together to build upon our knowledge and heighten our intellectual stimulation and explore the full potentials of data.

In her novel How We Think: Digital Media and Contemporary Technogensis, N Katherine Hayles claims E-literature representation “allows for criticisms and analysis of programmed media without sacrificing the interpretive strategies evolved with and through print” (Hayles 2012, 25). Thus, in accordance with Hayles’s description, the analysis of science articles through electronic forms of literature helps to enhance the strategies through which we can interpret the data. The brain interprets various media differently, thus the brain can be rewired through exposure to alternative kinds of media. The same data expressed in different forms of media can produce various outputs from the same input. Everyone has different experiences and perception is an embodied experience; as such, as the number of ways content can be experienced increases, the number of subsequent different perceptions that arise from the particular exposures increase as well. This increase in perceptions expands our thinking and widens the boundaries through which information can be extracted from data. Each media has benefits and limitations to what it can portray, yet having multiple media representing a single type of data works to extend the barriers of interpretations. “When humanities scholars turn to digital media, they confront technologies that operate on vastly different time scales, and in significantly different cognitive modes, than human understanding…the meshing of the these two different kinds of complex temporalities does not happen all at one time (or all at one place) but rather evolves as a complex syncopation between conscious and unconscious perceptions for humans…cross-connecting machine processes with human responses” (Hayles 2012, 13). Subsequently, crossing data with visual media links visualization with comprehension, thus increasing the cognitive modes that humans utilize for interpretation of the piece.

8. Conclusion

Augmenting text opens our minds to more cognitive pathways, thus increasing the ways in which we can interpret and share information. Specifically, scientific journal articles are the gateway to discovering cures and transferring scientific data between minds. Our minds augment the data as the information is interpreted, thus by broadening the range of potential perceptions increases the possibility of what can be extracted from the data. As the number of ways we can think increases, the boundary of what we can discover increases as well, removing the confines of text and allowing for our minds to interact together to actively understand the complexities of the world we live in.

Citations

 Hayles, N Katherine. How We Think:Digital Media and Contemporary Technogenesis London: University of Chicago Press, 2012. Print.

Iorio, Marilena V, Manuela Ferracin, et al. “MicroRNA Gene Expression Deregulation in Human Breast Cancer”. American Journal of Caner Research 65 (2005): 7065. Web. <http://cancerresaacrjournals.org/content/65/16/7065full

Manovich, Lev (2011) “What is Visualization?”, Visual Studies, 26: 1, 36-49.

“Structural consequences of cancer-causing BRCA1-BRCT missense mutation”. Protein Data Base RCSB .Web <http://www.rcsb.org/pdb/explore/remediatedSequencedo?structureId=1N5O&bionumber=1>

 van’t Veer, Laura J , et al. “Gene expression profiling predicts clinical outcome of breast cancer”. Nature.  41531 (2002): n page Web. <http://wwwnaturecom/nature/journal/v415/n6871/pdf/415530apdf>

 Yang, Shuan-Ping, San-Tai Song, and Hai-Feng Song. “Advancements of antisense oligonucleotides in treatment of breast cancer” .Acta Pharmacol Sin 244 (2003): n page Web. <http://www.chinaphar.com/1671-4083/24/289htm>

Media Tools

Voyant Tools Word Count Fountain: http://voyeurtoolsorg/tool/WordCountFountain

Text to Image: http://wwwteddavisorg/text2image/

 Wordle: http://wwwwordlenet/

Young-Hae Chang Heavy Industries: http://www.yhchang.com/THE_SEA.html

Featured Image: http://informationng.com/wp-content/uploads/2012/12/Wang-cancer-cell-.jpg