New robot platform improves patient experience using AI to help patients navigate barriers and health care challenges
SAN FRANCISCO, Sept. 12, 2019 /PRNewswire/ — Catalia Health and Pfizer today announced they have launched a pilot program to explore patient behaviors outside of clinical environments and to test the impact regular engagement with artificial intelligence (AI) has on patients’ treatment journeys. The 12-month pilot uses the Mabu® Wellness Coach, a robot that uses artificial intelligence to gather insights into symptom management and medication adherence trends in select patients.
The Mabu robot can interact with patients using AI algorithms to engage in tailored, voice-based conversations. Mabu “talks” with patients about how they are feeling and helps answer questions they may have about their treatment. The Mabu Care Insights Platform then delivers detailed data and insights to clinicians at a specialty pharmacy provider to help human caregivers initiate timely and appropriate outreach to the patient. The goal is to help better manage symptoms and address patient questions in real-time.
“At Catalia Health we’ve seen firsthand the benefits that AI has brought to healthcare for both the patient and the healthcare systems,” said Cory Kidd, founder and CEO of Catalia Health. “Our work with Pfizer allows us to engage with patients on a larger scale and therefore gain access to more insights and data that we hope can improve health outcomes.”
Mabu is helping to deliver personalized care by gaining insights that allow the specialty pharmacy to reach out to patients as they express challenges in managing their conditions. Mabu also generates health tips and reminders to help patients get additional information about their condition and treatment that may help them along the way. Over time, it is our goal that Mabu can help patients navigate barriers and health care challenges that are often a part of managing a chronic disease.
“The healthcare system is overburdened, and as a result, patients often seek more-coordinated care and information. Through this collaboration with Catalia Health, we hope to learn through real-time data and insights about challenges patients face, outside the clinical setting, with the goal to improve their treatment journeys in the future,” said Lidia Fonseca, Chief Digital and Technology Officer at Pfizer. “This pilot is an example of how we are working to develop digital companions for all our medicines to better support patients in their treatment journeys.”
The pilot program was officially announced on stage at the National Association of Specialty Pharmacy’s Annual Meeting & Expo on September 10, 2019. Initial pilot data will be available in the coming months. For more information, visit www.cataliahealth.com
About Catalia Health
Catalia Health is a San Francisco-based patient care management company founded by Cory Kidd, Ph.D., in 2014. Catalia Health provides an effective and scalable solution for individuals managing chronic disease or taking medications on an ongoing basis. The company’s AI-powered robot, Mabu, enables healthcare providers and pharmaceutical companies to better support patients living with chronic illness. Mabu uses a voice-based interface designed for simple, intuitive use by a wide variety of patients in remote care environments. The cloud-based platform delivers unique conversations to patients each time they have a conversation with Mabu.
Catalia Health’s care management programs are tailored to increase clinically appropriate medication adherence, improve symptom management and reduce the likelihood that a patient is readmitted to the hospital after being discharged.
For more information, visit www.cataliahealth.com
Jellyfish are about 95% water, making them some of the most diaphanous, delicate animals on the planet. But the remaining 5% of them have yielded important scientific discoveries, like green fluorescent protein (GFP) that is now used extensively by scientists to study gene expression, and life-cycle reversal that could hold the keys to combating aging. Jellyfish may very well harbor other, potentially life-changing secrets, but the difficulty of collecting them has severely limited the study of such “forgotten fauna.” The sampling tools available to marine biologists on remotely operated vehicles (ROVs) were largely developed for the marine oil and gas industries, and are much better-suited to grasping and manipulating rocks and heavy equipment than jellies, often shredding them to pieces in attempts to capture them.
Now, a new technology developed by researchers at Harvard’s Wyss Institute for Biologically Inspired Engineering, John A. Paulson School of Engineering and Applied Sciences (SEAS), and Baruch College at CUNY offers a novel solution to that problem in the form of an ultra-soft, underwater gripper that uses hydraulic pressure to gently but firmly wrap its fettuccini-like fingers around a single jellyfish, then release it without causing harm. The gripper is described in a new paper published in Science Robotics.
“Our ultra-gentle gripper is a clear improvement over existing deep-sea sampling devices for jellies and other soft-bodied creatures that are otherwise nearly impossible to collect intact,” said first author Nina Sinatra, Ph.D., a former graduate student in the lab of Robert Wood at the Wyss Institute. “This technology can also be extended to improve underwater analysis techniques and allow extensive study of the ecological and genetic features of marine organisms without taking them out of the water.”
The gripper’s six “fingers” are composed of thin, flat strips of silicone with a hollow channel inside bonded to a layer of flexible but stiffer polymer nanofibers. The fingers are attached to a rectangular, 3D-printed plastic “palm” and, when their channels are filled with water, curl in the direction of the nanofiber-coated side. The fingers each exert an extremely low amount of pressure – about 0.0455 kPA, or less than one-tenth of the pressure of a human’s eyelid on their eye. By contrast, current state-of-the-art soft marine grippers, which are used to capture delicate but more robust animals than jellyfish, exert about 1 kPA.
The researchers fitted their ultra-gentle gripper to a specially created hand-held device and tested its ability to grasp an artificial silicone jellyfish in a tank of water to determine the positioning and precision required to collect a sample successfully, as well as the optimum angle and speed at which to capture a jellyfish. They then moved on to the real thing at the New England Aquarium, where they used the grippers to grab swimming moon jellies, jelly blubbers, and spotted jellies, all about the size of a golf ball.
The gripper was successfully able to trap each jellyfish against the palm of the device, and the jellyfish were unable to break free from the fingers’ grasp until the gripper was depressurized. The jellyfish showed no signs of stress or other adverse effects after being released, and the fingers were able to open and close roughly 100 times before showing signs of wear and tear.
“Marine biologists have been waiting a long time for a tool that replicates the gentleness of human hands in interacting with delicate animals like jellyfish from inaccessible environments,” said co-author David Gruber, Ph.D., who is a Professor of Biology and Environmental Science at Baruch College, CUNY and a National Geographic Explorer. “This gripper is part of an ever-growing soft robotic toolbox that promises to make underwater species collection easier and safer, which would greatly improve the pace and quality of research on animals that have been under-studied for hundreds of years, giving us a more complete picture of the complex ecosystems that make up our oceans.”
The ultra-soft gripper is the latest innovation in the use of soft robotics for underwater sampling, an ongoing collaboration between Gruber and Wyss Founding Core Faculty member Robert Wood, Ph.D. that has produced the origami-inspired RAD sampler and multi-functional “squishy fingers” to collect a diverse array of hard-to-capture organisms, including squids, octopuses, sponges, sea whips, corals, and more.
“Soft robotics is an ideal solution to long-standing problems like this one across a wide variety of fields, because it combines the programmability and robustness of traditional robots with unprecedented gentleness thanks to the flexible materials used,” said Wood, who is the co-lead of the Wyss Institute’s Bioinspired Soft Robotics Platform, the Charles River Professor of Engineering and Applied Sciences at SEAS, and a National Geographic Explorer.
“At the Wyss Institute we are always asking, ‘How can we make this better?’ I am extremely impressed by the ingenuity and out-of-the-box thinking that Rob Wood and his team have applied to solve a real-world problem that exists in the open ocean, rather than in the laboratory. This could help to greatly advance ocean science,” said Wyss Institute Founding Director Donald Ingber, M.D., Ph.D., who is also the Judah Folkman Professor of Vascular Biology at Harvard Medical School, the Vascular Biology Program at Boston Children’s Hospital, and Professor of Bioengineering at SEAS.
The team is continuing to refine the ultra-soft gripper’s design, and aims to conduct studies that evaluate the jellyfishes’ physiological response to being held by the gripper, to more definitively prove that they do not cause the animals stress. Wood and Gruber are also co-Principal Investigators of the Schmidt Ocean Institute’s “Designing the Future” project, and will be further testing their various underwater robots on an upcoming expedition aboard the research ship Falkor in 2020.
Additional authors of the paper are Clark Teeple, Daniel Vogt, M.S., and Kevin Kit Parker, Ph.D. from the Wyss Institute and Harvard SEAS. Parker is a Founding Core Faculty member of the Wyss Institute and the Tarr Family Professor of Bioengineering and Applied Physics at SEAS. The research was supported by the National Science Foundation, The Harvard University Materials Research Science and Engineering Center, The National Academies Keck Futures Initiative, and the National Geographic Society.
InSystems and Model Engineering Solutions jointly developed a Simulink model of an adaptive fleet of InSystems’ proANT collaborating transport robots. The goal was to capture the desired adaptive system behavior to more effectively deal with the typical goals and challenges of collaborative embedded system groups (CSGs). A fleet of robots has to react to dynamic changes in the policy of the manufacturing execution system or the number and nature of its members to safeguard its functionality. The consistent application of a model-based development process for automation systems offers a variety of benefits to deal with these challenges. First and foremost, the specification of the CSG in the form of executable models allows for a fully virtual simulated representation of the robot fleet. This provides a sound foundation to efficiently develop and maintain the actual system. To exploit the full potential, a model-based approach relies on the reusability of models and test beds throughout the different development phases. Secondly, the model-based development process profits from a fully integrated tool chain that highly automatizes associated development activities. These include requirements management, modelling, and simulation as well as integrated quality assurance tasks, most notably, model-based static analysis and requirements-based testing. Tools such as the MES Model Examiner® and the MES Test Manager® are beneficial in streamlining the process.
The post Modelling of a Transport Robot Fleet in Simulink appeared first on Roboticmagazine.
D400 Depth Cameras. Credit: Intel CorporationThe Intel RealSense cameras have been gaining in popularity for the past few years for use as a 3D camera and for visual odometry. I had the chance to hear a presentation from Daniel Piro about using the Intel RealSense cameras generally and for SLAM (Simultaneous Localization and Mapping). The following post is based on his talk.
Depth information is important since that gives us the information needed to understand shapes, sizes, and distance. This lets us (or a robot) know how far it is from items to avoid running into things and to plan path around obstacles in the image field of view. Traditionally this information has come from RADAR or LIDAR, however in some applications we can also get that from cameras. In cameras we often get depth from using 2 cameras for stereo vision.
The Intel RealSense Depth camera (D400 series) uses stereoscopic depth sensing to determine the range to an item. So essentially it has two cameras and can do triangulation from them for stereo. This sensor uses two infrared cameras for the stereo and then also has an RGB camera onboard. So you can get 4 data products from the sensor; RGB image, depth image, left infrared image, and right infrared image. Think of each image frame as a 3D snapshot of the environment, where each color (RGB) pixel also has a range value (depth) to the item that is in the image. The farther the items are from the camera the greater the range/depth error will be.
The D400 cameras have an infrared projector for getting better features on surfaces with minimal texture in the infrared camera for computing the stereo reconstruction. This projector can be turned on and off if you want. Disabling the projector is often useful for tracking applications (since the projected dots don’t move with the items being tracked).
One thing to be aware is that the infrared images are rectified (to make the images look the same in a common plane) in the camera, however the RGB camera image is not rectified. This means that if you want the depth and RGB images to line up well, you need to manually rectify the RGB image.
Pro Tip 1: The driver has a UV map to help map from the depth pixel to the RGB image to help account for the difference in image sizes. This lets you match the depth to RGB image data points better.
Pro Tip 2: If using the D435i (the IMU version), use the timestamps from the images and IMU to synchronize the two sensors. If you use system time (from your computer) there will be more error (partially due to weird USB timing).
Pro Tip 3: The cameras have Depth Presets. These are profiles that let you optimize various settings for various conditions. Such as high density, high accuracy, etc..
Pro Tip 4: Make sure your exposure is set correctly. If you are using auto exposure try changing the Mean Intensity Set Point (the setting is not where exposure is, it is under AE control, not the most obvious).
If you want to use manual exposure you can play with are Exposure setpoint and Gain constant. Start with the exposure setpoint, then adjust the gain.
You might also want to see this whitepaper for more methods of tuning the cameras for better performance.
Visual odometry is the generic term for figuring out how far you have moved using a camera. This is as opposed to “standard” odometry using things such as wheel encoders, or inertial odometry with a IMU.
RealSense T265 is a tracking camera that is designed to be more optimal for Visual Odometry and SLAM (wider field of view and not using infrared light). It can do SLAM onboard as well as loop closure. However, this camera is not able to return RGB images (since it does not have a RGB camera onboard) and the depth returned is not as good as the D400 series (and can be a little trickier to get).
Using both a RealSense D435i sensor and a RealSense T265 sensor can provide both the maps and the better quality visual odometry for developing a full SLAM system. The D435i used for the mapping, and the T265 for the tracking.
Intel provides the RealSense SDK2.0 library for using the RealSense cameras. It is Open Source and work on Mac, Windows, Linux, and Android. There are also ROS and OpenCV wrappers.
Click here for the developers page with the SDK.
Within the SDK (software development kit) it includes a viewer to let you view images, record images, change settings, or update the firmware.
Pro Tip 5: Spend some time with the viewer looking at the camera and depth images when designing your system so you can compare various mounting angles, heights, etc.. for the cameras.
The RealSense SDK (software development kit) has a few filters that can run on your computer to try and improve the returned depth map. You can play with turning these on and off in the viewer. Some of these include:
Within ROS there is a realsense-ros package that provides a wrapper for working with the cameras in ROS and lets you view images and other data in RVIZ.
ROS RealSense Occupancy Map package is available as an experimental feature in a separate branch of the RealSense git repo. This uses both the D400 and T265 cameras for creating the map.
For SLAM with just the D435i sensor, see here.
Pro Tip 6: You can use multiple T265 sensors for better accuracy. For example, if one sensor is pointed forward and another backwards; you can use the confidence values from each sensor to feed into a filter.
I know this is starting to sound like a sales pitch
Pro Tip 7: If you have multiple cameras you can connect and query for a serial number to know which cameras is which.
Also if you remove the little connector at the top of the camera you can wire and chain multiple cameras together to synchronize them. (This should work in the SDK 2.0)
See this whitepaper for working with multiple camera configurations. https://simplecore.intel.com/realsensehub/wp-content/uploads/sites/63/Multiple_Camera_WhitePaper04.pdf
Pro Tip 8: Infinite depth points (such as points to close or to far from the sensor) have a depth value of 0. This is good to know for filtering.
Here are two code snippet’s for using the cameras provided by Daniel, to share with you. The first one is for basic displaying of images using python. The second code snippet is using OpenCV to also detect blobs, also using python.
## License: Apache 2.0. See LICENSE file in root directory.
## Copyright(c) 2015-2017 Intel Corporation. All Rights Reserved.
###############################################
## Open CV and Numpy integration ##
###############################################
import pyrealsense2 as rs
import numpy as np
import cv2
cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)
# Configure depth and color streams
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
# Start streaming
pipeline.start(config)
try:
while True:
# Wait for a coherent pair of frames: depth and color
frames = pipeline.wait_for_frames()
depth_frame = frames.get_depth_frame()
color_frame = frames.get_color_frame()
if not depth_frame or not color_frame:
continue
# Convert images to numpy arrays
depth_image = np.asanyarray(depth_frame.get_data())
color_image = np.asanyarray(color_frame.get_data())
# Apply colormap on depth image (image must be converted to 8-bit per pixel first)
scaled_depth=cv2.convertScaleAbs(depth_image, alpha=0.08)
depth_colormap = cv2.applyColorMap(scaled_depth, cv2.COLORMAP_JET)
# Stack both images horizontally
images = np.hstack((color_image, depth_colormap))
# Show images
cv2.imshow('RealSense', images)
k = cv2.waitKey(1) & 0xFF
if k == 27:
break
finally:
# Stop streaming
pipeline.stop()## License: Apache 2.0. See LICENSE file in root directory.
## Copyright(c) 2015-2017 Intel Corporation. All Rights Reserved.
###############################################
## Open CV and Numpy integration ##
###############################################
import pyrealsense2 as rs
import numpy as np
import cv2
def nothing(args):
pass
def detectBlobs(mask):
# Set up the SimpleBlobdetector with default parameters.
params = cv2.SimpleBlobDetector_Params()
# Change thresholds
params.minThreshold = 1;
params.maxThreshold = 255;
# Filter by Area.
params.filterByArea = True
params.maxArea = 4000
params.minArea = 300
# Filter by Circularity
params.filterByCircularity = True
params.minCircularity = 0.1
# Filter by Convexity
params.filterByConvexity = True
params.minConvexity = 0.5
# Filter by Inertia
params.filterByInertia = True
params.minInertiaRatio = 0.1
detector = cv2.SimpleBlobDetector_create(params)
# Detect blobs.
reversemask= mask
keypoints = detector.detect(reversemask)
im_with_keypoints = cv2.drawKeypoints(mask, keypoints, np.array([]),
(0,0,255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
return im_with_keypoints
def thresholdDepth(depth):
depth[depth==0] = 255 #set all invalid depth pixels to 255
threshold_value = cv2.getTrackbarPos('Threshold','Truncated Depth')
# Zero if dist>TH
ret,truncated_depth=cv2.threshold(scaled_depth,threshold_value,255,cv2.THRESH_BINARY_INV)
return truncated_depth
cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)
cv2.namedWindow('Truncated Depth', cv2.WINDOW_AUTOSIZE)
cv2.createTrackbar('Threshold','Truncated Depth',30,255,nothing)
# Configure depth and color streams
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
# Start streaming
pipeline.start(config)
try:
while True:
# Wait for a coherent pair of frames: depth and color
frames = pipeline.wait_for_frames()
depth_frame = frames.get_depth_frame()
color_frame = frames.get_color_frame()
if not depth_frame or not color_frame:
continue
# Convert images to numpy arrays
depth_image = np.asanyarray(depth_frame.get_data())
color_image = np.asanyarray(color_frame.get_data())
# Apply colormap on depth image (image must be converted to 8-bit per pixel first)
scaled_depth=cv2.convertScaleAbs(depth_image, alpha=0.08)
depth_colormap = cv2.applyColorMap(scaled_depth, cv2.COLORMAP_JET)
# Stack both images horizontally
images = np.hstack((color_image, depth_colormap))
# Show images
cv2.imshow('RealSense', images)
truncated_depth=thresholdDepth(scaled_depth)
truncated_depth=detectBlobs(truncated_depth)
cv2.imshow('Truncated Depth', truncated_depth)
k = cv2.waitKey(1) & 0xFF
if k == 27:
break
finally:
# Stop streaming
pipeline.stop()I hope you found this informative and can make use of the Pro Tips.
Thank you to Daniel for presenting this information and allowing me to share it. This content is based on his talk. Daniel has also provided the full slide set that can be accessed by clicking here.
Disclaimer: I have not received any funding or free items from Intel.
Liked this article? Take a second to support me on Patreon! This post appeared first on Robots For Roboticists.