Fullscreen HTML5Med Res (??MB)HTML4

Artificial vision is becoming a reality The BIONIC EYE Vision is our most important sense, accounting for about 80% of information received by our brains. The loss of vision can therefore have a dramatic effect on a person, especially if they lose it through accident or disease. Now there is promising research on how to restore a basic sense of vision. A Finally, a method of human vision for the blind involving s with the well-established cochlear implant (“bionic ear”, of which Australia is a world leader) no hardware but just skill is also presented. which can restore a sense of hearing, there is now active research on how to restore a basic sense of vision, How the eye works In nature, ten different types of eye layout can be found. using an implantable visual prosthesis or “bionic eye”. So as not to give false expectations, artificial vision does The eye layout found in humans and vertebrate animals, cephalopods (squid and octopuses) and some spiders most not provide a visual experience like natural vision. Bionic vision is a popular theme in science fiction, two resembles a traditional camera. In this type of eye, called a camera-type eye, light enters notable examples being Colonel Steve Austin in “The Six Million Dollar Man” and Lieutenant Commander Geordi through the cornea which acts as a window and also refracts light like a meniscus lens, contributing two thirds of the La Forge in “Star Trek: The Next Generation” (see box). Like many other themes from science fiction, bionic optical power of the eye. It then passes through the iris which alters its diameter vision is also becoming a reality – even if the science is to adjust the amount of light entering the eye, then through in its infancy. Apart from implants to restore a sense of vision there the adjustable lens which adjusts its focal length to focus are also non-implanted prosthetic devices that work on the objects from different distances and then projects an image principle of sensory substitution whereby the sense of sight onto the light sensitive retina at the back of the eye. The eye lens also has a graded is converted into an alternative optical index (like modern optical sense such as touch or sound Part 1 - By Dr David Maddison fibres) for maximum efficiency and these will also be discussed. 28  Silicon Chip siliconchip.com.au and also contributes one third of the optical power of the eye. The retina Being the light-sensitive part of the eye, the retina is also the part which is often diseased or damaged, leading to a severe vision deficit or blindness. The retina is comprised of a number of layers containing neurons which communicate with each other via synapses. A neuron is an electrically active cell that can receive inputs, process them and produce an output. An output signal from a neuron is transmitted to other cells across a synapse (see SILICON CHIP, “Interfacing to the Brain”, January 2015). Some neurons are specialised as photoreceptor cells which are sensitive to light. The two main types of these specialised neurons are rods and cones. Rods are sensitive in low light and provide monochrome vision while cone cells are sensitive to colour and work in bright light. There are about 100,000,000 rods and 5,000,000 cone cells. Visual signals from the rods and cones are processed by other neurons in the retina to reduce the amount of visual data that has to be sent back to the brain. One of the ten layers of the retina is the ganglion cell layer. The rods and cones are connected to this layer via another type of neuron. The ganglion cells are a type of neuron that has a very long axon that extends from the eye back into the brain to form the optic nerve, optic chiasm and optic tract (see diagram and text below). It is the ganglion cells that carry information from the retina into the brain. There are about 1,500,000 ganglion cells. The axon is the part of the cell body that connects to other neurons and from which information leaves (See the above article from SILICON CHIP, January 2015, for discussion of Ganglion Cells Structure of a human eye. neurons and axons.) It is the rods and the cones, the photoreceptor cells which are most often damaged by disease while the underlying layers which carry visual information back to the brain such as the ganglion cell layer are usually left intact. These remaining layers can be used to introduce visual information to the brain via a prosthetic retinal implant in one type of bionic eye. Note from above there are around 105,000,000 rod and cone cells generating information and only 1,500,000 ganglion cells to convey that information back to the brain. This lack of a one to one correspondence is suggestive of the amount of data processing that has occurred in the eye itself. To complicate matters further, the surface of the retina is not uniform in its properties or photoreceptor density. Bipolar Cells (red) Horizontal Cell Amacrine Cells (blue) Photoreceptors Simplified cross section of a human retina. Counter-intuitively, light enters on the left of the diagram which is the inner part of the eye. This is the ganglion cell layer which is the connecting circuitry that takes information back to the brain. The light then travels toward the photoreceptors (rods and cones) which are at the outer part of the eye where light is converted to signals which are transmitted to the ganglion cells through several layers. This is the opposite arrangement to an imaging chip in a camera whereby the light sensitive elements are at the light receiving side and the connecting circuitry is beneath that. In the various layers, points of light from the rods and cones are processed to identify features such as movement, simple shapes, edges and bright points surrounded by dark points before the information is sent back to the brain for further processing. Image credit: “Retina layers1”. Licensed under CC BY-SA 3.0 via Wikipedia – http:// en.wikipedia.org/wiki/File:Retina_layers1.gif#/media/File:Retina_layers1.gif siliconchip.com.au June 2015  29 Image from the retina, showing higher resolution image from fovea, the small part of the retina responsible for the sharpest vision. The brain fills in for the rest of the retina which produces a less sharp image but is not noticed under normal circumstances. Frame grab from https://youtu.be/ 4I5Q3UXkGd0 The fovea is a small part of the retina, about 1.5mm in diameter, that is responsible for sharp vision, with a very high concentration of cone cells. The fovea is connected to about half of the nerve fibres in the optic nerve while the rest of the retina connects to the other half. The fovea represents only about 1% of the retinal surface but uses 50% of the visual cortex, showing its great importance in sight. Its visual field is small, equivalent to about two thumbnails at arm’s length so to get a sharp image of an object the eye has to scan back and forth to build up an image. The fovea also only has cone cells so is not sensitive at night. The blind spot You’ll probably remember those biology lessons at school where a card is moved in and out from one eye and at some point an “X” on the card disappears. That is caused by the blind spot, an area where the optic nerve passes through the retina and no photoreceptors exist. (See www.education. com/science-fair/article/eye-retinal-blind-spot/). Normally the brain makes up for the lack of receptors in that area so its effect is not noticed. The shape and size of the eye are also important considerations for bionic prostheses. The eye is not spherical but it is roughly the shape of two hemispherical sections joined together. Also, despite people coming in all shapes and sizes, the size of the eye between different individuals is remarkably uniform and is around 24mm front to back varying by only up to 2mm. This means perhaps only one size of visual prosthetic device that goes in the eye (or replaces it) will ever need to be manufactured. It has been estimated that the data bandwidth of the human eye is 8.75Mbits/s. The neurons could fire much faster giving a much higher speed but there is a trade-off of speed and energy and data processing efficiency. A question often asked and which is important for comparing natural vision to a bionic eye is what is the resolution of the human eye. It is not simple to answer that question because unlike a still camera, the eye does not record a static image. The eye records a video stream of sorts but the neural hardware of the eye and brain extract and see only that information that is relevant, somewhat like highly compressed 30  Silicon Chip video data where only changes in a picture are transmitted. The question is further complicated by the fact that the eye and body move and the brain assembles these images from different viewpoints into a type of composite image that has more information than the number of photosensitive cells in the eye would suggest (like taking a number of still images panning across a scene and assembling them into a larger image). Taking all of the above into account, one conservative estimate made by Roger Clark (www.clarkvision.com/ articles/human-eye/) for the resolution of the human eye is 576 megapixels to view a scene of 120° x 120° but the real field of view is even larger than this. Other estimates are that what we see is equivalent to the high resolution area of the fovea having a 7-megapixel resolution and the rest of the eye having a resolution of one megapixels. These issues are discussed in the video “What Is The Resolution Of The Eye?” https://youtu. be/4I5Q3UXkGd0 Structure of the visual system The visual system of an advanced organism such as a mammal usually consists of the following principal components: • the eye and its main component containing photo receptors, the retina; • the optic nerve for relaying information from the retina to the brain; • the optic chiasm which causes signals from the optic nerves to partially cross to allow the visual cortex to receive a complete visual field from both eyes and then combine them for stereoscopic vision; • the lateral geniculate body which has multiple functions and receives information from the retina via the optic nerve and optic chiasm and also processes that data before passing it on via the optic radiations to the visual cortex where the sense of vision is generated. Visual system of a human. Note how the visual fields represented by the green and orange colours start in the retina, partially cross at the optic chiasm and are finally mapped onto the visual cortex. siliconchip.com.au Even though the eye has the same basic optical elements as a camera as described above, it is far more than a camera and a lot of processing of visual data is done inside the retina itself with numerous different types of neurons involved as well as processing of visual data done elsewhere in the brain. Function of the bionic eye A bionic eye works by stimulating some part of the visual system in order to generate a sense of vision in cases where the eye or other components of the visual system are absent, diseased or defective. As the nervous system and brain use electric currents to convey information, electrical stimulation is the obvious choice to stimulate the visual system. Historical background The use of an electrical current to stimulate vision was first undertaken in 1755 by Frenchman Charles LeRoy who passed electricity through the eye of a blind man and this resulted in the him perceiving the sensation of light. Following that was the discovery of electrical activity in animal brains in 1875 but this involved exposing their brains which was a procedure not amenable to human experiments. The first EEG or electroencephalograph to record these brainwaves was taken of a dog in 1914 by Hans Berger who invented that machine. At the end of WWI in 1918 the first observations were made in Germany that electrical stimulation of the surface of the visual cortex in patients undergoing neurosurgical procedures under local anaesthesia resulted in the patient seeing dots of light or “phosphenes”. In 1924 Hans Berger recorded the first electrical activity from a human brain with scalp electrodes, a remarkable achievement at the time given the small voltages involved and the recording instruments of the time. Otfrid Foerster in 1929 investigated electrical stimulation of the occipital lobe (where the visual cortex is located) and reported that people could see a dot of light. The idea that many sites could be simultaneously stimulated to provide vision was postulated by W. Krieg in 1953. Of course, the complex electronics required to drive multiple electrode arrays in a portable package would not be available from some decades not to mention suitable implant materials. The measurement of electrical activity in the brain and its connection to visual processes was thus established leading to the possibility of artificial vision for the blind as well as a large array of other possibilities for interfacing the human brain to machines; see Interfacing to the Brain, SILICON CHIP, January 2015. Eye diseases and conditions to be treated Two common causes or visual impairment or blindness are among conditions sought to be treated with bionic vision: • Age-related macular degeneration is a condition resulting in the loss of central vision leading to the loss of abilities such as reading, facial recognition, reading clocks and street signs. Peripheral vision is maintained although the area of central vision loss gets larger with time. The fovea, responsible for high resolution vision, is part of the macula. • Retinitis pigmentosa is a degenerative condition of the siliconchip.com.au The bionic eye in science fiction The Six Million Dollar Man was a 1973 TV series which featured a bionic man, Colonel Steve Austin, with a bionic eye. What was portrayed as a fantasy 42 years ago, appears to be within the grasp of current or foreseeable technology. Also, Star Trek: The Next Generation featured Lieutenant Commander Geordi La Forge with a bionic eye. Catalog description of The Six Million Dollar Man’s bionic eye. Screen grabs from https:// vimeo.com/ 77027616 CAD diagram, very good for 1973 vintage, showing The Six Million Dollar Man’s bionic eye and interface circuitry to the visual cortex. The bionic eye of The Six Million Dollar Man. Lieutenant Commander Geordi La Forge from Star Trek: The Next Generation with his VISOR device (Visual Instrument and Sensory Organ Replacement) that can see most of the electromagnetic spectrum. It is interfaced to his brain via the optic nerves. The technology for this type of device seems a little further into the future than that of The Six Million Dollar Man’s device. June 2015  31 Representation of a parked car at different resolutions. In order of increasing resolution these images are 16 (4x4), 64 (8x8), 144 (12x12), 256 (16x16), 1024 (32x32), 4096 (64x64) and 16384 (128x128) pixels. Note that these images indicate the amount of information that might be conveyed at a particular resolution, not what a person would necessarily see. These images are also grey scale. Retinal and cortical prostheses currently display phosphenes (pixels) that are either off or on with no shades or colours. Also, in a current retinal or cortical implant, individual pixels will have space between them. The sensory substitution device, The vOICe does have 16 shades of “loudness”. (Courtesy Dr Peter Meijer, The vOICe.) eye due to the loss of photoreceptor cells and an increasing loss of peripheral vision resulting in tunnel vision and eventual blindness. In both the above cases, the photoreceptor cells have died but the neural pathway to the brain remains intact so in principle, this pathway can be activated with a retinal implant that stimulates the remaining pathway. Vision loss due to missing eyes or optic nerve damage can be treated by stimulation of areas such as the lateral geniculate body or the visual cortex within the brain. Ways of interfacing a bionic eye to the brain Consideration of the anatomy of the human visual system as described above suggests four ways a bionic eye can interface to the brain. An account needs to be made of the fact that the retina itself processes information and so does the lateral geniculate body and the visual cortex. The further along the visual pathway one goes before an interface is made it would seem that the more complicated it would be to make an effective prosthesis as the device might have to generate more “processed” visual data and less “raw” data. On the other hand, neuroplasticity, the ability of the brain to rewire itself might assist in developing a workable interface to any implanted prosthetic device. 1) As stated above, when disease affects the retina, it mainly destroys the photoreceptor cells leaving the ganglion cell layer, which transmits data to the brain, intact. Interfacing a device with this layer would therefore seem to be an effective way to interface a prosthetic device. Exceptions are if the retinal disease is so severe that even ganglion cells are destroyed or there is damage to the optic nerve. There are several locations within the retina where an implant can be located. Epiretinal implants are located on the inner surface of retina, subretinal implants are located behind the retina and suprachoroidal implants are located above the choroid and behind the retina. 2) Beyond the ganglion layer of the retina, there is a possibility of interfacing with the optic nerve although this involves challenges due to accessibility issues and also interfacing to a thin nerve with around about 1,000,000 nerve fibres. One such example is the Microsystem-based Visual Prosthesis (MIVP) which consists of a spiral cuff electrode wrapped around the optic nerve. Unlike retinal or cortical implants which produce monochrome phosphenes, coloured phosphenes have been reported in this stimulation method. Test subjects have also been able to locate and discriminate between objects. 3) Interfacing to structures such as the lateral geniculate body deep within the brain is possibly risky and complicated although this is a site being researched for interfacing a bionic eye. At the lateral geniculate body the visual data has not yet been so extensively processed that it has become too complicated to interpret and map. At this point a visual scene is mapped onto the brain tissue in a relatively simple way and bears a correspondence to the scene being observed. It has been estimated that the maximum resolution of an electrode array implanted at the location would be 40x40 per side. 4) The final interfacing possibility is the primary visual cortex of the brain (V1) which is close to the surface of the brain and relatively accessible. This area is specialised for processing information about stationary and moving objects and pattern recognition. The visual image of the retina is mapped onto V1 and a large portion of that retinal map corresponds to the fovea. Stimulation of this region of the brain enables a person to generate points of light (phosphenes) which can be used to generate a form of vision as has already been shown in experiments. A problem with using V1 as an interface is that the mapping of the retina is not linear so that, say, a square electrode area would not correspond to the same shape in the visual field. The first experiments in artificial vision The three possible locations of retinal implants. (Courtesy Bionic Vision Australia.) 32  Silicon Chip In the early 1960s Giles Brindley and W.S. Lewin in the UK started researching artificial vision and this resulted siliconchip.com.au in 1968 of the implant of 80 electrodes into the visual cortex of a blind person. The experiment was a success and the subject was able to identify letters and patterns in the phosphenes that were generated by electrical stimulation of the 80 electrodes and the research was published in a classic scientific paper in 1969. This lead to a major international conference at the University of Chicago which was to establish future directions for this work. Giles Brindley’s work inspired numerous similar research projects in the 1970s with the main objective of assisting the blind to read with the low resolution image provided by 80 or so electrodes. Many experiments were done stimulating the visual cortices of volunteers who were having neurosurgery for other reasons as well as volunteers having electrode implants. It soon became less important to assist the blind to read due to the development of talking books recorded on cassette tape and the emphasis became that of assisting the blind to navigate in their environment. This required a portable electronic package to do the visual processing required to create a usable image on the implanted electrodes but at the time creating a small portable processing unit was not possible with the electronics available; this technology would not be available until the 1990s. Jeremiah Teehan is credited by the Guinness Book of Jeremiah Teehan, the man who had the world’s first artificial vision system. Unfortunately, the implant deteriorated and had to be removed. The cortical implant is shown in image (a) and an x-ray of the implant in (b) the glasses/ camera combination is shown in (c) and the processing unit in (d). (From “Organic Bionics”, Wiley-VCH, 2012). siliconchip.com.au Records as the first person to have an artificial eye. The device was developed by the late William Dobelle and others. The record is dated 17th January 2000 and he had 68 platinum electrodes implanted on the surface of the visual cortex of his brain, although only 20 worked effectively and gave a narrow field of view, and he wore glasses containing a camera and an ultrasonic rangefinder as well as a 4.5kg visual processor unit on a shoulder strap. He had vision the equivalent of a severely short-sighted person with 20/400 vision and saw the outline of objects and letters. Unfortunately the implant deteriorated and had to be removed. The support electronics could be substantially miniaturised today. Another of William Dobelle’s patients, Jens Naumann, wrote an account of his experience with artificial vision called “Search for Paradise: A Patient’s Account of the Artificial Vision Experiment”. Also, see the video “Jens Naumann: Artificial Vision” https://youtu.be/JWMYW-SkURI His implant also deteriorated and he is again blind. Also see pictures of his implant at www.jensnaumann. green-first.com/gallery.shtml Early experiments with bionic vision as described above involved electrode arrays on the visual cortex but one alternative approach was to stimulate the retina itself. The first clinical trial of a 16 electrode retinal implant was made in 2002 by Second Sight Medical Products: www.2sight.com What was once only a dream of restoring vision in the blind has now progressed to a reality today with people actually using visual prostheses that give them some visual perception of the world. Desired resolution It should be noted that the objective of bionic eye research is not to provide the equivalent of natural vision as this is way beyond any technology currently available, but as with the cochlear implant, it is designed to give a workable, usable replacement for a lost or missing sense which may have much less fidelity than the natural equivalent but can still be of tremendous help to the person using the technology. An important question to answer is: what resolution of image is usable for a blind person to navigate about the world, say to walk to shops or catch public transport and read signs and food labels? This question applies equally to either a bionic eye or a sensory substitution device. It has been demonstrated in studies that a resolution of 32x32 pixels or 1024 pixels is more than enough to get meaningful and usable images. At lower resolutions a 4x4 array will provide motion detection capability, an approximately 100 electrode array will provide a navigational capability and an approximately 1,000 electrode array will provide facial and letter recognition. A video showing different resolutions of retinal implant can be seen at https://youtu.be/4gaBAIzAn-M [Project Xense Retinal Implant Simulation]. Note the separation between SC individual pixels. NEXT MONTH: In the final part of this mini series, we will look at some of the amazing advances being made here in Australia in the quest for the perfect Bionic Eye. June 2015  33