Virtual Vegas @ DASL-UNLV will use emerging technologies to create a functional robot that provides an immersive experience at a remote location that supports multi-modal interaction in scenarios of real-world utility. The motivation of the team is to create remote workers in order to manifest the “future of work.” The future of work consists of a changing environment in which technology will replace certain jobs. Many workers are currently misrepresented in the current workforce, and people with disabilities are unlikely to be a part of the current workforce. Since people with disabilities make up 12.7% of the US population, there is a large benefit toward making the workplace more inclusive towards these individuals. If robots will become part of the workforce, then through technological advances, a worker can be “present” on the job while not having to physically be at the location of the job. The goal of the team will be to use haptics, humanoid robots, VR/AR technology, and BCI robot control to make this dream into a reality. Remote workers can wear EEG headsets that will control robotic movement, while a haptic glove will provide force feedback from the humanoid robot. Moreover, the use of EMG will be used to control the hand of the humanoid intuitively as if it were your own, and with minimal effort. The VR/AR technology will allow the operator to be immersed in a 3D visual and audio surroundings, providing excellent situational awareness and the fulfillment and self-efficacy provided from a typical in-person job.
Virtual Vegas @ DASL-UNLV is a diverse team with a wide variety of experience in different skills. The team is experienced with tackling hard problems in robotics research, led by one of the leading professors in humanoid robotics research, Dr. Paul Oh. Some members of the team have experience competing in the DARPA robotics challenge where the DRC-HUBO@UNLV team placed in 8th place with the same humanoid that will be used for the Avatar Xprize Competition. Since then, the team has focused on acquiring different skills from machine learning to haptics to aerial manipulation. The team is also made up of different cultures and experiences, providing a wide variety of viewpoints on outcomes and approaches to developing these technologies. Our team has members that are experts in VR/AR, humanoid robotics, computer vision, BCI, robot control, and many other methods and technologies. Members of the team have experience working with platforms such as: Spot Mini from Boston Dynamics, DRC-Hubo from Rainbow Robotics, ROBOTIS-OP from ROBOTIS, and the Optitrack motion capture system, among others. Multiple members of the team have spent time researching at the Naval Research Laboratory, Korea Advanced Institute of Science and Technology, and the Jet Propulsion Laboratory. This wide variety of skill sets, experiences, and viewpoints will make Virtual Vegas @ DASL-UNLV a competitive and impactful team during the ANA Avatar XPRize competition.
The technology proposed by the team will not only push the boundaries of robotics research, but will revolutionize the future of robotic avatars. The platform developed will make lives easier by connecting people around the world. A person in Las Vegas will be able to control the robot anywhere in the world through the technology developed by this team. Furthermore, these technologies will advance society toward the “future of work,” providing self efficacy to millions of disabled individuals across the world and creating millions of jobs at the same time. The state of EEG/BCI research will be advanced with a newly developed dry EEG headset which has a higher resolution and is more comfortable to wear for a long duration. The headset will be accurate in high level control of multiple robotic platforms in multiple applications, including the control of the robotic avatar. The team is working to develop 3D visual and auditory sensing similar to what humans are able to experience in any environment. The operator will be able to have a first person and third person point of view of the robotic avatar, which will allow them to gain better situational awareness. This will advance the current state of robot teleoperation, allowing for novel methods of robot operation, and opening the ability to new robot applications in many different industries. Moreover, by creating low-latency frameworks for control and computer vision, it will be easier to control robots remotely, greatly advancing the current state of robot control.
The robotic avatar that will be used for the competition will be DRC-Hubo, a humanoid robot designed and built by the team's partner lab in South Korea, KAIST Hubo Lab, and in conjunction with the research performed by the team’s institution, the Drones and Autonomous Systems Lab (DASL). The robot contains 32 degrees of freedom giving it plenty of freedom to perform many actions. During the 2015 DARPA Robotics Challenge Finals, the humanoid was able to handle a power drill, drive a Polaris ATV, open doors, turn valves, handle power connectors, walk on uneven terrain, climb stairs, and clear rubble. Since humanoid robots resemble humans, they are able to perform nearly any action that a human can perform, as long as algorithms and methods can be developed to simulate human performance. These actions were demonstrated in the 2015 DARPA Robotics Challenge. The team also has research experience testing with DRC-Hubo in scenarios such as dynamic lift-and-carry of different materials, coordination with human coworkers, and VR teleoperation of the robot for disaster relief. For the ANA Avatar Xprize competition, the humanoid robot will be able to move omnidirectionally in walking mode, allowing the avatar to navigate around complex objects and maintain a dynamic footprint as humans can. Additionally, the robot contains another mode that allows it to move around using differential drive with wheels driven by motors mounted to the knees of the robot. This allows for the robot to only use computational resources for balance during necessary scenarios such as navigating uneven terrain and climbing stairs, and the robot can wheel around when on normally even surfaces for ease of use. DRC-Hubo is also equipped with over-actuated robot arms that allow the robot to manipulate a wide variety of objects based on end-effector position and orientation. The new hand that will be designed for DRC-Hubo will have force sensing capabilities in its fingers, allowing for real-time force reflection to the user during operation of the robotic avatar. The team envisions the avatar being used as a remote worker. For example, the humanoid robot can go into a disaster scene like the Fukushima Nuclear Disaster and be controlled by someone in another country from a safe and monitored location. In addition to this, the operator will be able to have full situational awareness of the robot regardless of where it is located by using VR/AR technologies in conjunction with 3D scene reconstruction technology. Finally, the robot will have the ability to be controlled by a EEG/BCI headset. In this scenario, the platform can be used by both able bodied and non able bodied people, opening new horizons for disabled person to control robots across the world from any location in real-time. Combining all of these control frameworks, sensor modalities, and robotic platforms will allow for the creation of a robotic avatar that can operate remotely in real-time while providing an immersive experience and efficient control by multiple levels of operators for a wide variety of applications and services.
Virtual Vegas @ DASL-UNLV has access to state-of-the-art facilities in a wide variety of areas including humanoid robotics, virtual reality, unmanned autonomous vehicles, and service robotics. Our team works in the Drones and Autonomous Systems Lab (DASL) at the University of Nevada, Las Vegas (UNLV). DASL is housed within a X,000 sq. ft. building near the main campus of UNLV, and thus has large space available to test the avatar in a wide variety of scenarios. DASL's facilities include:
2 motion capture systems (1 large for drone testing, and 1 small for humanoid robots)
VR space with associated equipment (e.g. VR headset and controllers)
Humanoid robot testing area with 1 large motorized gantry, 3 mobile gantry units, and a variety of testing scenarios (valve turning, metal staircase, door opening)
Woodshop with laser cutter, screenprinter, and woodworking tools
Machine shop with enclosed CNC mill, PCB manufacturing jig, and metalworking/welding tools
Electronics shop with multiple 3D printers, soldering equipment, and electronic components
Augmented reality & projection mapping room with controllable light levels
1 functional full-sized humanoid robot (DRC-Hubo) & 1 social full-sized humanoid robot (Jaime Hubo)
3 minitaure humanoid robots (ROBOTIS-OP1/2), 3 co-bot arms (1 HDT & 2 RB5), 1 service robot (Furo), 1 multimedia robot (Jay)
A large stock of various UAV types and sizes and associated control, service, and development equipment