Jean Mary Zarate: 00:05
Hello, and welcome to Tales From the Synapse, a podcast brought to you by Nature Careers in partnership with Nature Neuroscience. I’m Jean Mary Zarate, a senior editor at the journal Nature Neuroscience, and in this series we speak to brain scientists all far more than the globe about their life, their investigation, their collaborations, and the impact of their operate.
In episode five, we meet a researcher devoted to understanding the complexities of vision and how to bring eyesight back to the blind.
Pieter Roelfsema: 00:40
So my name is Pieter Roelfsema. I’m the director of the Netherlands Institute for Neuroscience in Amsterdam. And I’m also a professor at the Amsterdam University Healthcare Hospital, and at the No expense University of Amsterdam.
I studied the visual brain due to the truth quite a few years, and I regularly wanted to know what happens if you essentially commence to recognize what you see.
And that is far from trivial, for the explanation that basically, what you see is just quite a few merchandise, quite a few small image elements that fall on your retina.
But then you have to sort of group that with each and every other into a representation of objects and the numerous objects that might possibly surround you. And that is a extremely fascinating subject.
So I studied that for, for quite a few years. And in the final couple of years, we found ourselves implanting an escalating quantity of electrodes (and electrodes is just an further word for wires) into the brain of experimental animals.
And at some point, we realized we are at such a larger quantity, hundreds, let’s ramp it up a modest bit.
So we went to 1000, with the notion of stimulating these electrodes and then making artificial vision.
Pieter Roelfsema: 01:56
So I really feel we have numerous ambitions. 1st, is essentially to recognize how vision functions. And 1 of the subjects that I’m especially interested in is interest.
So of all the components that surround you, you can invest interest, select out 1 of these merchandise, what happens in the brain when you direct your interest to some factor.
And an further element is what happens if some factor enters into your conscious awareness? So what are extremely fascinating inquiries, so some components might possibly get into your awareness and some, some do not. So we are also seeing that.
And 1 of the further applied ambitions of the lab is to create a visual prosthesis, a visual brain prosthesis, so persons who lost the function in their eyes, the notion will be just to skip these malfunctioning eyes.
And to plug in the visual information from a camera. It is 1 of the central centres, so centres for vision in the brain.
So we strategy that from numerous angles. So 1 is to do modelling. Second is to study vision in humans, just possessing them respond with a button press, understanding what they can and cannot see.
But we also seem at the brain mechanisms in experimental animals. So we seem in mice, and we also seem in monkeys.
Pieter Roelfsema: 03:20
So I started to turn into interested in the brain appropriate right after reading a book by Doug Hofstadter. It is, I really feel it is a renowned book. It is referred to as Gödel, Escher, Bach, and was extremely renowned when I was starting to study in the ‘80s.
It was a present from my father. And when I study it, I knew what I wanted to do, I wanted to study consciousness. So then, I basically started to do some projects just in my hobby time, initial on snails, then on rodents. And at some point, I realized I want to study this in an animal that is closer to humans. And I will essentially recognize what happens in the brain when we turn into consciously conscious of some factor.
So we, in my lab, studying cognition and the function of interest and how it is modulating the activity in the visual brain, we started to implant escalating numbers of electrodes. So electrodes are just wires.
And so, at some point, we reached a standard preparation specifically exactly where we implant, say, 200 electrodes, and then we believed, you know, we could multiply this with a small element. And then we know that from preceding operate, that if you stimulate 1 electrode electrically, you take place to be artificially activating these brain cells close to the tip of the electrode. And a specific particular person or an experimental animal (it can even be a specific particular person who has been blind for further than ten years), they will see a dot of light. And that is with only 1 electrode.
So if you have one particular hundred or 1000 electrodes, you can create 1000 of these dots of lights, for the explanation that phosphines. And due to the truth the region, the visual cortex, specifically exactly where we implant these electrodes, have a map of space, specifically exactly where you stimulate in a map the subject sees at the equivalent location, the outdoors visual globe, this dot of light.
So if you have 1000, you can fundamentally operate with them like a matrix board, they can know from the stadium or from the highway. So if you of course, if you flash up 1 bulb, the specific particular person is going to see, correctly, a dot of light. But you can create patterns. And that is what we set out to do.
So we’re fundamentally writing to this matrix board that is in the brain, and, and see no matter whether or not the animals are capable to recognize them. These, these components that we build as patterns. And we found surely that this is the case.
So we have been capable to build, for instance, we educated, we did this in monkeys. We educated them to recognize letters.
And so they knew that if they saw letter A they would have to make an eye movement to the above. If they saw the letter B, to the left, and so on and so forth.
And at some point we educated them visually so these animals have been not blind, they could see.
At some point, we took the visual stimulus away, and we just wrote straight letter A to the brain.
And we have been extremely excited to uncover that they have been surely creating the equivalent response, as when we would have presented the equivalent letter visually. We published that in 2020. So 1-and-a-half years ago.
So in our group, we call for a lot of diverse expertise. And some of these sorts of expertise are inside our individual group. So we are knowledgeable about how to location wires, electrodes in the brain.
But we also have quite a few collaborations with specialists about the globe, persons who essentially know about how to make these electrodes so that they do not harm the brain tissue as properly substantially.
We operate with persons in artificial intelligence who assistance us to take our camera, to take camera pictures and translate them into brain stimulation patterns.
We also collaborate with neurosurgeons who can inform us how to essentially make this device and make it some factor that is going to be feasible for a neurosurgeon to essentially implant in the brain for the explanation that that is undoubtedly a extremely crucial aim for me, to bring this to a patient.
So the visual procedures will be composed of numerous components. The initial is a camera. You can use the camera that you obtain. So there are now numerous firms that make these glasses that incorporate a camera.
And these camera pictures are sent to a small laptop or computer system. It can be the size of a phone. And this will take in the camera image and create from it a pattern to be a post on the matrix boards in the brain.
Then correct now, we nonetheless have essentially a physical connection in amongst a connector that is implanted on the skull of the subject. It can be a monkey. It can be a human. We would like to make this wireless, so it will be a wireless interconnects with a brain chip.
And then from the brain chip, there will be numerous wires operating into the brain. So these are the ones we make contact with electrodes. And so, mostly primarily based on the image that the camera captures, there is this brain stimulation pattern.
And that then supplies a rudimentary form of vision. So you take place to be not going to see total colour, total depth as frequent vision would give you. It is going to be extremely rudimentary, like you are walking about with this massive matrix board in front of you, correct? So it is, it is undoubtedly not going to be great, but it is practically surely also going to be substantially far much better than absolutely nothing at all.
Yes, eyesight functions. It starts, of course, all in the retina, that is at the back of the eye, which is a extremely, extremely sophisticated device.
So there are important groups of researchers that are studying the retina. And then from there, the information is transported to the brain by indicates of the optic nerve. And then it starts in the cortex in the initial region, principal visual cortex.
And there are cells, brain cells, neurons, that are selected for comparatively straightforward traits of the outdoors globe, say the location and the orientation of an edge of light, no matter whether or not that is a vertical edge or a horizontal edge.
And so they essentially do a extremely neighborhood processing. So you have quite a few of these processors and in parallel. So 1 would be straight ahead, 1 would be just adjacent, 1 would be in the upper left corner for each and every and each location in the outdoors visual globe, there is a set of neurons that just care about what is going on there.
And then if you go to higher regions, then this information of these particular person detectors is combined in further and further sophisticated approaches. So fundamentally, what happens there is that you go from pixels, to concepts. And there are now quite a few persons modelling this.
So there also has been, of course, an artificial intelligence revolution that helped us recognize how to go from pixels to concepts. And what these persons in artificial intelligence uncover and how they model this strategy basically, is pretty a pretty excellent approximation of what is going on in the human brain.
So also, in the human brain you have all these stages that are involved in this translation from pixels into concepts.
Now, seeing what the concept in front of you is, no matter whether or not it is a bicycle or a chair, that is only 1 of the functions of vision, it is not the only 1. You also can steer your motor behaviour. And there are other brain regions that are involved in that. So they basically localize the edges. So if you want to select some factor up, you call for to know specifically exactly where your fingers are going to touch the object that you want to select up.
And you call for to know specifically exactly where it is, you call for to know how to position all your joints, all your joints. And so all these transformations, they are also in aspect informed by vision. So that is an further extremely crucial function for vision to play.
Pieter Roelfsema: 11:42
So 1 element that is, I really feel, fascinating in this domain is the possibility now to at times record also neurons from human people, and some researchers in my lab are performing this.
So these are people who have intense forms of epilepsy. And the neurologist does not uncover the correct cocktail to suppress these epileptic attacks. So then the neurosurgeon comes into the play, and in some situations it is clear what is the problematic region of the brain, but there are some occasional situations specifically exactly where the neurosurgeon is not one particular hundred% confident.
And then these people get the set of electrodes, a set of wires in their brain, for about two weeks. And we have ethical approval then to attach to these clinical electrodes, extremely tiny wires.
And by indicates of these, we can record single neurons. So that was a tactic that was produced by Itzhak Fried numerous years ago. That supplies you the unique opportunity to also record from brain cells that are tuned to distinct persons.
So if you record from these single neurons, you can also do exceptional components. So other persons, but also in our lab, from time to time you can, for instance, make associations in amongst stimuli.
So suppose that, while you associate a renowned specific particular person, say Jennifer Aniston, with an further renowned specific particular person, say, Barack Obama, then we demonstrated that if persons recall these associations…so you give them a image of Jennifer Aniston, and you ask them to recall what was associated with them, then you have some neurons that only respond to Barack Obama. And then they will turn into active the moment you speak about Jennifer Aniston, and ask them to recall this association.
So these components I’m also extremely excited about. For the explanation that the neurons that code these concepts are commonly also promptly the concepts that are basically in your consciousness, these are the components you take place to be pondering about.
So that supplies you a extremely close hyperlink to what is essentially on the subject’s thoughts. And what you can see in the activity of neurons, which I uncover fascinating.
Yeah, so we are not recreating the eye. So we’re just skipping it. So I really feel that is also why the vision that we’re going to build is, it is just substantially substantially significantly less major top quality than the frequent vision.
For the explanation that we are implanting electrodes in the particular person brain, and if we stimulate these, we activate a set of neurons that would typically beneath no situations be activated in that constellation, That supplies you just a dot of lights.
And it does not give us the possibility to create diverse colours, for instance, for the explanation that neurons that are selected for diverse colours are intermingled, and you cannot just selectively only activate the green cells, or the blue cells, or red cells. So that is why it is somewhat rudimentary.
But the challenges if you sort of realize that you take place to be beneath no situations going to be as pretty excellent as frequent vision, then are to get a pretty excellent coverage of the visual fields for the explanation that of the nature of the map of the outdoors globe in the brain.
You have to realize that the principal visual cortex, which is the initial region specifically exactly where the information comes from, the visual information that is processed in the cortex is massive. It has a surface area of 25 square centimetres on the left, and an further 25 square centimetres on the correct.
And to get wires everywhere in that region, which is also pretty folded, is going to be hard. So that is 1 of the large challenges that we’re pondering about, how to make confident that we cover the map with electrodes.
If you only cover a small aspect of the map with electrodes, then the subject is only going to be capable to see in a small region of the visual field, then they will be blind at all other regions. That is pretty undesirable.
One particular far more large challenge is to make an interface with electronics in the brain that has a sufficient longevity.
So we are now at the moment using so-referred to as Juta electrode arrays. So these are arrays of stiff silicone shanks, we make contact with them. So like, like a bed of nails, is fundamentally what it seems like.
And we know that they operate, typically for a year, probably a modest bit longer. But you know, you do not want to implant a patient with a prosthetic device to uncover out that appropriate right after 1 or two years these electrodes are encapsulated by glial cells. So fundamentally the fibrosis, fibrous tissue that encapsulates the electrodes, and you shed the make make contact with with with the nerve cells. So in that case, you cannot successfully stimulate any longer.
So that is an further challenge. We have to uncover electronic elements that have sufficient longevity. So if you implant them now, they will nonetheless be functioning, say, in five years, or ten years, or even 15 years. I really feel these are two most important challenges.
Pieter Roelfsema: 16:58
So I get the occasional request. And I have to clarify to these persons who make make contact with with me, this is not a clinically authorized device. So it is investigation. And our ambition will be to go to humans in the subsequent say, two years, or probably a modest bit later.
But in that case, it is nonetheless going to be investigation. So do not count on from us in the coming five years a therapy. It is just investigation. And of course, the investigation is extremely crucial for the explanation that it is going to assistance us make the subsequent step, and go towards a device that is clinically authorized.
Ahead of we are there there are all sorts of regulations, which are there for a pretty excellent objective. And we have to show that we comply with all these regulations.
With the technologies we’re using now, it is regularly going to be rudimentary. But I would be extremely excited if you are capable to create a prosthesis that has, say 1000, or even ten,000, or even 50,000 pixels, ought to realize that your eye has 1 million pixels.
So if you count the quantity of fibres in the optic nerves, it is about 1 million. So 50,000 is what we might properly aim for at some point. Sounds ambitious, but it is only 5% of the frequent, of the frequent eye. And it is, that is going to be hard, but if I would seem back on my profession, and we would have been capable to create a device that has 50,000 pixels, and numerous persons are using it and it is catching up, I will be tremendously satisfied about it.
Jean Mary Zarate: 18:51
Now that is it for this episode of Tales From the Synapse. I’m Jean Mary Zarate, a senior editor at Nature Neuroscience. The producer was Don Byrne. Thanks when once more to Professor Pieter Roelfsema. And thank you for listening.