Centimeter-Level People Tracking: 35×35m Venue | Marvelmind

0:01 Hello colleagues, let's talk today about another very typical case, which is people tracking, but not in warehouses or assembly plants, but more in exhibitions, museums, offices—let's say less industrial areas. It may seem to be an easier task, but in fact it is not. The complexity stems from the complexity of the area. It could be a small area but much more complex in terms of obstructions, in terms of external noise, in terms of unpredictability. This particular case is a bit special because there was only one mobile object, which is pretty unusual because typically you need to track dozens of people in the same area, like visitors.

0:58 Of museums or similar, but in this particular case it was only one, but very special. So let's talk about the case in detail. What is it? It was about very precise tracking of a guide into an exhibition—a special exhibition. The guide would be followed by a virtual avatar—basically a projection on the wall that would follow you when you walk through the exhibition. The exhibitor would be showing and telling the story about the exhibits, and the virtual avatar would complement and walk next to their human. So this is why, unlike in our previous case when the

1:57 latency wasn't so important but the accuracy was—in this case we should have met both high accuracy and at the same time low latency. Unfortunately, they cannot share much more about the case, but overall, the stuff I just explained, and you can see typical tracking and typical behavior in this video. So you have a floor plan—okay, that was the old video. However, you have a badge, which is the mobile tracking device on the person. It could be another type of mobile beacon. It could be installed on a tablet, for example, or it could even be a jacket if it's a special

2:59 exhibition. But in the majority of cases, a badge is recommended. Since this video was shot a couple of years ago, now we have an improved and updated version of the badge. So that was the recommended way for the mobile beacon. So you have a badge, you have a map, and you have a mobile beacon. As you see in this case, the latency was around two seconds. The accuracy and the overall performance is about the same, but the latency in this case wasn't the task and it was relatively high. In this newer case, the latency was around half a second with the real-time player enabled. Without the real-time player enabled, it could be even less—it would be a quarter of a second. So effectively, with real-time player you can choose between more robust tracking and

3:59 higher latency at the same time, or lower latency and increased accuracy. For example, to even half a second or even two seconds. In this case it would be very smooth and very robust, like in this case as well. So what are the problems when you deploy the system like this? Of course, it's indoor positioning first of all, because there are not too many solutions that can meet several requirements. What are those requirements? First of all, high accuracy of course. Marvelmind indoor GPS is specializing in high precision indoor positioning. In this case, ultrasound band would be a nice option because they didn't require centimeter level. Let's say 10 to 30 centimeters that ultrasound band can provide was sufficiently good for them. In some cases, optical could be an option, but in this particular case it was pretty tough because there was flasher lights and

4:58 you know, most of the time it was a dark exhibition. Very loud music could be a problem for our system because it's acoustic based, but luckily it wasn't, because again, the system is designed to withstand loud acoustic noise unless it goes to ultrasound. But even for ultrasound, we have many ways to combat those noises. What other problems? Of course, obstructions. Line of sight, line of sight, line of sight is a must. This is why we employ several techniques to combat the obstruction. But in general, obstruction and non-line of sight is probably the biggest problem in any indoor positioning system—for our case, but as well for UWB and of course for optical as well. So in one way or another, you do need to prevent non-line of sight obstruction

5:56 by putting more beacons or by employing some other techniques. So what is the solution? Solution is very typical in terms of used equipment. Stationary beacons are Super-Beacons installed on the walls, high on the walls. Installing higher on the walls allows their lower level, lower probabilities of obstruction. The Modem, the central controller of the system—additionally, since there were many walls and some of these walls, even though they were glass walls, they were coated with a thin metal wire—it wasn't radio transparent walls. So the radio coverage was an issue at some point in time. This is why we had to install full-size antennas and place the Modem in an optimal place in the center, just to make sure that the radio coverage

6:55 is not an issue at all. Even though the dimensions of the territory was relatively small—it was just 35 by 35 meters—but as I mentioned in the beginning, in many cases smaller territory doesn't mean the network is actually simpler. This is the new badge, and of course it's based on Mini-RX. The improved badge has a split omni inside the strap. So on the left side and on the right side there are microphones, and you can put the strap in one way or another. At least one of the microphones would be facing outwards on each side of the neck. So it meant that your own neck or your own body wouldn't create a non-line of sight situation. And this is why this is one of the methods we strongly suggest for a non-line of sight situation

7:53 when the chances are high. So use badge, for example. In this case, as usually, our system is based on time-of-flight of ultrasound to the stationary beacons and calculating the position of the mobile beacon using trilateration. It's a highly precise method of calculating. This is why we suggested every time when high precision is required. So this is a very typical performance before any kind of real-time player or any smoothing or any filtering applied. Oh, actually filtering is applied. For example, here you don't see some of the location updates now, simply because the system was not able to determine the location and filters it out. We could provide the raw data, or actually we do provide the raw data. So your system can pick up the raw data and apply your own smoothing mechanism or

8:52 filtering mechanism, or even you know, try to estimate the position based on sum data. For example, distances. This is location, but distances to the stationary beacons we do provide. So it means that you can employ that data and try to recover the locations even in difficult non-line of sight situations. So this is a typical view inside our Dashboard. We cut all the details outside, so this is kind of in the area. So as you can see, it has 11 submaps—relatively small exhibition, 35 by 35 meters, but 11 submaps. Why? Because it's complex. This is the mobile beacon. This is the track of the mobile beacon. The update rate was around 4.9 hertz, so 5 hertz. The beacons are placed. Now you'll see on the next screens the beacons are placed all over the

9:50 area in order to provide the main requirement. The main requirement in order to position the mobile beacon: the mobile beacon must be heard or must hear two or more stationary beacons with direct line of sight within 30 meters away. We deployed the Inverse Architecture and we deployed fully overlapping TDMA submaps. Let's look at them. So this is their view on the same territory but with the floor plan uploaded to the system. Okay, in this case there was up to 12, or actually 13 submaps because we start from zero, and the real-time player enabled. So it's much, much smoother, as you say. So those omissions in their tracking are not visible anymore. But the real-time player brings an additional latency, as said. So for example if you have TDMA2.

10:49 And this TDMA2 brings—it doesn't increase the latency, but the real-time player does. And for example, it first collects their location data of four measurements, and only then shows. So effectively I have around a latency of four. So if I have four Hertz or five Hertz, then the latency would be four divided by four, seven, and one Hertz, so about one second latency. It was okay for this because it wasn't too fast moving. But you could reduce their TDMA, or sorry, real-time player level to two or three. It will be less smooth, but the latency would be decreased proportionally. Or you can increase the smoothness even further. So it can tolerate even longer jumps or longer emissions, but the latency will be longer. So it's

11:49 up to you. And of course we stream out raw data after real-time player and so forth. So you can choose depending on your needs. Now let's talk about the most details about the system. As you see, the system is pretty complex. What is the complexity? As I mentioned, complexity comes from the complexity of the area. So there's the entry and the exhibition goes. Let's say the exhibitor goes this way. And there are a lot of obstacles. There are exhibits which are floating in the air. Thus, for example, when you're in this area, this beacon which would be covering in one of the TDMA slots wouldn't be seen in this because there will be a clear no line of sight situation for this at this moment.

12:47 Beacon 4 and beacon 6 would be serving, whereas beacon 7 wouldn't be possible. But when, for example, sometimes people are surrounding your exhibitor, and then none of these beacons are seen, let's say seen clearly. So this is why you may have a no line of sight situation. But again, we combat it with two ways—two major ways, actually. One is the badge. The badge has a split omni microphone on the left side of your neck and on the right side of your neck. It's on the neck, so it means that the chances that you obstruct with your own body are low. For example, compared to a beacon which you can put in your chest pocket. Now in this case, half of the time your own body would obstruct the beacons

13:47 from the back. Now this is possible, but not really recommended, because in this case the tracking would be more poor and you would need to install even more beacons in order to provide the coverage. So this white badge is the more optimal solution. The second major optimal, let's say, method to combat their non-line of sight situation is full overlapping submaps. So for example, beacon number four and beacon number six are covering this area, and beacon number five and beacon number seven are covering exactly the same area. Why? Now, if you are facing this wall, then these beacons would be tracking you. Yes, it would be tracking only half of the time because the other half, because the split between this time division multiple access submaps is 50/50. So one, two, one, two, one, two, one, two. But at least you would be covered 50 percent of the time. And

14:44 the real-time player would smooth the missed location updates. If you are in a good shape and, let's say, visible by beacon 4 and 6, 5 and 7, then there will be no skipped or missed location updates, and you will be tracked either from the left or from the right. From the left, from the right. And if there is a minor discrepancy in location from this submap to this submap, then the real-time player once again would smooth this. But usually you align during the network planning so that there will be no jumps, no like, you know, this kind of zigzag type of behavior. No, it's pretty, pretty straightforward, even without. Like you have seen here, for example, there's no jumps between the location updates even though this and this were measured by the opposite submaps on the opposite walls.

15:42 So what else? It's very important to see the service zones. Service zones are basically defining where their submap is responsible for a particular tracking. For example, a submap consisting of beacon four and six—this is the serving service zone for this submap. And for five and seven, this is the service zone for this submap. So they are overlapping. And it's pretty tricky to make these arrangements because when their area is large, you can reuse the same frequencies over the distance. So for example, distances between these beacons are less than 30 meters. And if it's less than 30 meters, their signal from this beacon may fly to this area and may basically interrupt

16:40 the tracking. Luckily, it flies after the signal of this beacon, so we basically neglect it because it's already too late. We are getting the signal either from this beacon or from this beacon, and the signal that flies in comes already later when we have determined the location based on this. But nevertheless, it's pretty complex because too many beacons are condensed in the small area and, you know, thorough network planning is required. So this is why for smaller maps consisting of, you know, one, two submaps, our customers can do this by themselves. But for more complex maps like this one, we recommend that we do it because again, we are far more experienced and we can do this much faster than you can. So if your case is similar to this—for example, tracking of people

17:40 on the exhibitions or all kinds of conferences or museums, and you may be interested in this, please don't hesitate to contact us and send us an email to info@marvelmind.com or visit our website www.marvelmind.com, and we would be happy to answer your questions and offer the most optimal solution for your particular case. Thank you very much.