Skip to main content

2024 | OriginalPaper | Buchkapitel

A Color Event-Based Camera Emulator for Robot Vision

verfasst von : Ignacio Bugueno-Cordova, Miguel Campusano, Robert Guaman-Rivera, Rodrigo Verschae

Erschienen in: Robotics, Computer Vision and Intelligent Systems

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Event-based cameras are becoming increasingly popular due to their asynchronous spatial-temporal information, high temporal resolution, power efficiency, and high dynamic range advantages. Despite these benefits, the adoption of these sensors has been hindered, mainly due to their high cost. While prices are decreasing and commercial options exist, researchers and developers face barriers to addressing the potential of event-based vision, especially with more specialized models. Although accurate event-based simulators and emulators exist, their primary limitation lies in their inability to operate in real-time and in that they are designed only for grey-scale video streams. This limitation creates a gap between theoretical exploration and practical application, hindering the seamless integration of event-based systems into real-world applications, especially in robotics. Moreover, the importance of color information is well recognized for many tasks, and most existing event-based cameras do not handle color information, except for a few exceptions. To address this challenge, we propose a ROS-based color event camera emulator to aid in reducing the gap between the real-world applicability of event-based color cameras by presenting its software design and implementation. Finally, we present a preliminary evaluation to demonstrate its performance.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
3.
Zurück zum Zitat Almatrafi, M., Baldwin, R., Aizawa, K., Hirakawa, K.: Distance surface for event-based optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 42(7), 1547–1556 (2020)CrossRef Almatrafi, M., Baldwin, R., Aizawa, K., Hirakawa, K.: Distance surface for event-based optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 42(7), 1547–1556 (2020)CrossRef
4.
Zurück zum Zitat Bajestani, S.E.M., Beltrame, G.: Event-based RGB sensing with structured light. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 5458–5467 (2023) Bajestani, S.E.M., Beltrame, G.: Event-based RGB sensing with structured light. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 5458–5467 (2023)
5.
Zurück zum Zitat Brandli, C., Berner, R., Yang, M., Liu, S.C., Delbruck, T.: A 240\(\times \) 180 130 db 3 \(\mu \)s latency global shutter spatiotemporal vision sensor. IEEE J. Solid-State Circ. 49(10), 2333–2341 (2014)CrossRef Brandli, C., Berner, R., Yang, M., Liu, S.C., Delbruck, T.: A 240\(\times \) 180 130 db 3 \(\mu \)s latency global shutter spatiotemporal vision sensor. IEEE J. Solid-State Circ. 49(10), 2333–2341 (2014)CrossRef
6.
Zurück zum Zitat Buber, E., Diri, B.: Performance analysis and CPU vs GPU comparison for deep learning. In: 2018 6th International Conference on Control Engineering & Information Technology (CEIT), pp. 1–6 (2018) Buber, E., Diri, B.: Performance analysis and CPU vs GPU comparison for deep learning. In: 2018 6th International Conference on Control Engineering & Information Technology (CEIT), pp. 1–6 (2018)
7.
Zurück zum Zitat Finlayson, G.D.: Colour and illumination in computer vision. Interface focus 8(4), 20180008 (2018)CrossRef Finlayson, G.D.: Colour and illumination in computer vision. Interface focus 8(4), 20180008 (2018)CrossRef
8.
Zurück zum Zitat Furmonas, J., Liobe, J., Barzdenas, V.: Analytical review of event-based camera depth estimation methods and systems. Sensors 22(3), 1201 (2022)CrossRef Furmonas, J., Liobe, J., Barzdenas, V.: Analytical review of event-based camera depth estimation methods and systems. Sensors 22(3), 1201 (2022)CrossRef
9.
Zurück zum Zitat Gallego, G., et al.: Event-based vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(1), 154–180 (2022)CrossRef Gallego, G., et al.: Event-based vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(1), 154–180 (2022)CrossRef
10.
Zurück zum Zitat Gehrig, M., Aarents, W., Gehrig, D., Scaramuzza, D.: Dsec: a stereo event camera dataset for driving scenarios. IEEE Robot. Autom. Lett. 6(3), 4947–4954 (2021)CrossRef Gehrig, M., Aarents, W., Gehrig, D., Scaramuzza, D.: Dsec: a stereo event camera dataset for driving scenarios. IEEE Robot. Autom. Lett. 6(3), 4947–4954 (2021)CrossRef
12.
Zurück zum Zitat Hu, Y., Liu, S.C., Delbruck, T.: v2e: From video frames to realistic DVS events. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1312–1321 (2021) Hu, Y., Liu, S.C., Delbruck, T.: v2e: From video frames to realistic DVS events. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1312–1321 (2021)
13.
Zurück zum Zitat Kaiser, J., et al.: Towards a framework for end-to-end control of a simulated vehicle with spiking neural networks. In: 2016 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots, pp. 127–134 (2016) Kaiser, J., et al.: Towards a framework for end-to-end control of a simulated vehicle with spiking neural networks. In: 2016 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots, pp. 127–134 (2016)
14.
Zurück zum Zitat Kalluri, T., Pathak, D., Chandraker, M., Tran, D.: Flavr: flow-free architecture for fast video frame interpolation. Mach. Vis. Appl. 34(5), 83 (2023)CrossRef Kalluri, T., Pathak, D., Chandraker, M., Tran, D.: Flavr: flow-free architecture for fast video frame interpolation. Mach. Vis. Appl. 34(5), 83 (2023)CrossRef
15.
Zurück zum Zitat Kessler, C., et al.: Programmability and performance portability aspects of heterogeneous multi-/manycore systems. In: 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1403–1408 (2012) Kessler, C., et al.: Programmability and performance portability aspects of heterogeneous multi-/manycore systems. In: 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1403–1408 (2012)
16.
Zurück zum Zitat Li, C., et al.: Design of an RGBW color VGA rolling and global shutter dynamic and active-pixel vision sensor. In: 2015 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 718–721. IEEE (2015) Li, C., et al.: Design of an RGBW color VGA rolling and global shutter dynamic and active-pixel vision sensor. In: 2015 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 718–721. IEEE (2015)
17.
Zurück zum Zitat Lichtsteiner, P., Posch, C., Delbruck, T.: A 128\(\times \)128 120 db 15 \(\mu s\) latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circ. 43, 566–576 (2008)CrossRef Lichtsteiner, P., Posch, C., Delbruck, T.: A 128\(\times \)128 120 db 15 \(\mu s\) latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circ. 43, 566–576 (2008)CrossRef
18.
Zurück zum Zitat Long, X., Gong, X., Zhang, B., Zhou, H.: Deep learning based data prefetching in CPU-GPU unified virtual memory. J. Parallel Distrib. Comput. 174, 19–31 (2023)CrossRef Long, X., Gong, X., Zhang, B., Zhou, H.: Deep learning based data prefetching in CPU-GPU unified virtual memory. J. Parallel Distrib. Comput. 174, 19–31 (2023)CrossRef
19.
Zurück zum Zitat Marcireau, A., Ieng, S.H., Simon-Chane, C., Benosman, R.B.: Event-based color segmentation with a high dynamic range sensor. Front. Neurosci. 12, 317614 (2018)CrossRef Marcireau, A., Ieng, S.H., Simon-Chane, C., Benosman, R.B.: Event-based color segmentation with a high dynamic range sensor. Front. Neurosci. 12, 317614 (2018)CrossRef
20.
Zurück zum Zitat Moeys, D.P., et al.: A sensitive dynamic and active pixel vision sensor for color or neural imaging applications. IEEE Trans. Biomed. Circuits Syst. 12(1), 123–136 (2017)CrossRef Moeys, D.P., et al.: A sensitive dynamic and active pixel vision sensor for color or neural imaging applications. IEEE Trans. Biomed. Circuits Syst. 12(1), 123–136 (2017)CrossRef
21.
Zurück zum Zitat Niklaus, S., Mai, L., Liu, F.: Video frame interpolation via adaptive convolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2270–2279 (2017) Niklaus, S., Mai, L., Liu, F.: Video frame interpolation via adaptive convolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2270–2279 (2017)
22.
Zurück zum Zitat Nikolić, G.S., Dimitrijević, B.R., Nikolić, T.R., Stojcev, M.K.: A survey of three types of processing units: CPU, GPU and TPU. In: 2022 57th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), pp. 1–6 (2022) Nikolić, G.S., Dimitrijević, B.R., Nikolić, T.R., Stojcev, M.K.: A survey of three types of processing units: CPU, GPU and TPU. In: 2022 57th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), pp. 1–6 (2022)
23.
Zurück zum Zitat Quigley, M., et al.: ROS: an open-source robot operating system. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Workshop on Open Source Robotics. Kobe, Japan (2009) Quigley, M., et al.: ROS: an open-source robot operating system. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Workshop on Open Source Robotics. Kobe, Japan (2009)
24.
Zurück zum Zitat Rebecq, H., Gehrig, D., Scaramuzza, D.: ESIM: an open event camera simulator. In: Billard, A., Dragan, A., Peters, J., Morimoto, J. (eds.) Proceedings of The 2nd Conference on Robot Learning. Proceedings of Machine Learning Research, vol. 87, pp. 969–982. PMLR (2018) Rebecq, H., Gehrig, D., Scaramuzza, D.: ESIM: an open event camera simulator. In: Billard, A., Dragan, A., Peters, J., Morimoto, J. (eds.) Proceedings of The 2nd Conference on Robot Learning. Proceedings of Machine Learning Research, vol. 87, pp. 969–982. PMLR (2018)
25.
Zurück zum Zitat Sawant, A., Saha, A., Khoussine, J., Sinha, R., Hoon, M.: New insights into retinal circuits through EM connectomics: what we have learnt and what remains to be learned. Front. Ophthalmol. 3, 1168548 (2023)CrossRef Sawant, A., Saha, A., Khoussine, J., Sinha, R., Hoon, M.: New insights into retinal circuits through EM connectomics: what we have learnt and what remains to be learned. Front. Ophthalmol. 3, 1168548 (2023)CrossRef
26.
Zurück zum Zitat Shah, P., Rathod, S.S.: Review of bio-inspired silicon retina: from cell to system level implementation. In: 2021 International Conference on Communication information and Computing Technology (ICCICT), pp. 1–13. IEEE (2021) Shah, P., Rathod, S.S.: Review of bio-inspired silicon retina: from cell to system level implementation. In: 2021 International Conference on Communication information and Computing Technology (ICCICT), pp. 1–13. IEEE (2021)
27.
28.
Zurück zum Zitat Skorych, V., Dosta, M.: Parallel CPU-GPU computing technique for discrete element method. Concurrency Comput. Pract. Experience 34(11), e6839 (2022)CrossRef Skorych, V., Dosta, M.: Parallel CPU-GPU computing technique for discrete element method. Concurrency Comput. Pract. Experience 34(11), e6839 (2022)CrossRef
29.
Zurück zum Zitat Taverni, G., et al.: Front and back illuminated dynamic and active pixel vision sensors comparison. IEEE Trans. Circ. Syst. II Express Briefs 65(5), 677–681 (2018) Taverni, G., et al.: Front and back illuminated dynamic and active pixel vision sensors comparison. IEEE Trans. Circ. Syst. II Express Briefs 65(5), 677–681 (2018)
30.
Zurück zum Zitat Zhu, A.Z., Thakur, D., Özaslan, T., Pfrommer, B., Kumar, V., Daniilidis, K.: The multivehicle stereo event camera dataset: an event camera dataset for 3D perception. IEEE Robot. Autom. Lett. 3(3), 2032–2039 (2018)CrossRef Zhu, A.Z., Thakur, D., Özaslan, T., Pfrommer, B., Kumar, V., Daniilidis, K.: The multivehicle stereo event camera dataset: an event camera dataset for 3D perception. IEEE Robot. Autom. Lett. 3(3), 2032–2039 (2018)CrossRef
Metadaten
Titel
A Color Event-Based Camera Emulator for Robot Vision
verfasst von
Ignacio Bugueno-Cordova
Miguel Campusano
Robert Guaman-Rivera
Rodrigo Verschae
Copyright-Jahr
2024
DOI
https://doi.org/10.1007/978-3-031-59057-3_24

Premium Partner