{"id":2633,"date":"2020-12-06T11:04:38","date_gmt":"2020-12-06T02:04:38","guid":{"rendered":"https:\/\/www.codedojo.com\/?p=2633"},"modified":"2020-12-06T12:05:50","modified_gmt":"2020-12-06T03:05:50","slug":"using-computer-vision-to-enforce-sleeping-pose-with-the-jetson-nano-and-opencv","status":"publish","type":"post","link":"https:\/\/www.codedojo.com\/?p=2633","title":{"rendered":"Using computer vision to enforce sleeping pose with the Jetson Nano and OpenCV"},"content":{"rendered":"\n<figure class=\"wp-block-embed-twitter wp-block-embed is-type-rich is-provider-twitter\"><div class=\"wp-block-embed__wrapper\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\"><p lang=\"en\" dir=\"ltr\">Blog: Using computer vision to enforce sleeping pose with the Jetson Nano and OpenCV <a href=\"https:\/\/t.co\/pN4hTnAtnu\">https:\/\/t.co\/pN4hTnAtnu<\/a> <a href=\"https:\/\/t.co\/N6URzK3muF\">pic.twitter.com\/N6URzK3muF<\/a><\/p>&mdash; Seth A. Robinson (@rtsoft) <a href=\"https:\/\/twitter.com\/rtsoft\/status\/1335405189168201729?ref_src=twsrc%5Etfw\">December 6, 2020<\/a><\/blockquote><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script>\n<\/div><figcaption>(special thanks to Eon-kun for helping demonstrate what it looks like)<\/figcaption><\/figure>\n\n\n\n<p>Imagine you HAVE to sleep on your back for some reason and possibly restrict neck movement during the night.  Here are some options:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Tennis balls strapped to sides<\/li><li>Placing an iphone on chest\/pocket and using an app (<a href=\"http:\/\/www.proximalbox.com\/somnopose\/\">SomnoPose<\/a>) that monitors position with the accelerometer and beeps when it detects angle changes.  (it works ok but the app is old and has some quirks like not running in the background)<\/li><\/ul>\n\n\n\n<p>The above methods are missing something though &#8211; they don&#8217;t detect head rotation.  If you look at the wall instead of the ceiling while not moving your body, they don&#8217;t know. <\/p>\n\n\n\n<p>The tiny $99 Jetson Nano computer combined with a low light USB camera can solve this problem in under 100 lines of Python code! (A Raspberry Pi would work too)<\/p>\n\n\n\n<p>The open source software OpenCV is used to processed the camera images. When the program can&#8217;t detect a face, it <strong>plays an annoying sound<\/strong> until it does, forcing you to wake up and move back into the correct position so you can enjoy sweet silence.<\/p>\n\n\n\n<p>If you&#8217;re interested in playing with stuff like this, I recommend Paul McWhorter&#8217;s <a href=\"https:\/\/www.youtube.com\/watch?v=5INy0FvaWLw&amp;list=PLGs0VKk2DiYxP-ElZ7-QXIERFFPkOuP4_\">&#8220;AI on the Jetson Nano&#8221;<\/a> tutorial series, the code below can be used with that.<\/p>\n\n\n\n<p>I&#8217;m really excited about the potential of DIY electronics projects like this to help with real life solutions.  <\/p>\n\n\n\n<p>The Pi and Nano have GPIO pins so instead of playing a sound, we could just as easily activate a motor, turn a power switch on, whatever.<\/p>\n\n\n\n<p>Of course instead of just tracking faces, it&#8217;s also possible to look for eye, colors, shapes or cars, anything really.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The Python code listing for Forcing you to sleep on your back<\/h2>\n\n\n\n<pre class=\"wp-block-syntaxhighlighter-code\">import cv2\nimport time\nfrom playsound import playsound\nimport os\n\ndispW=1024\ndispH=768\ntimeAllowedWithNoFaceDetectBeforeWarning = 22\ntimeBetweenWarningsSeconds = 10\n\ntimeOfLastFaceDetect = time.time()\ntimeSinceLastDetect = time.time()\ntimeOfLastWarning = time.time()\nwarningCount = 0\n\ndef PlayWarningIfNeeded():\n    global timeBetweenWarningsSeconds\n    global timeOfLastWarning\n    global warningCount\n\n    if time.time() - timeOfLastWarning > timeBetweenWarningsSeconds:\n        print (\"WARNING!\")\n        warningCount = warningCount + 1\n        os.system(\"gst-launch-1.0 -v filesrc location=\/home\/nano\/win.wav ! wavparse ! audioconvert ! audioresample ! pulsesink\")\n        timeOfLastWarning = time.time()\n\n\nbCloseProgram = False\n\ncv2.namedWindow('nanoCam')\ncv2.moveWindow('nanoCam', 0,0)\ncam = cv2.VideoCapture(\"\/dev\/video0\")\n\ncam.set(cv2.CAP_PROP_FRAME_WIDTH,int(dispW))\ncam.set(cv2.CAP_PROP_FRAME_HEIGHT,int(dispH))\ncam.set(cv2.CAP_PROP_FPS, int(10))\nface_cascade = cv2.CascadeClassifier('\/home\/nano\/Desktop\/Python\/haarcascades\/haarcascade_frontalface_default.xml')\nfnt = cv2.FONT_HERSHEY_DUPLEX\n\nwhile True:\n\n    ret, frame = cam.read()\n    frame = cv2.flip(frame, 0) #vertical flip\n\n    #rotate 90 degrees\n    #frame = cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE)\n    gray=cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n    faces = face_cascade.detectMultiScale(gray, 1.3, 5)\n\n    for (x,y,w,h) in faces:\n           cv2.rectangle(frame, (x,y), (x+w, y+h), (0,255,0), 4)\n           timeOfLastFaceDetect = time.time()\n   \n    timeSinceLastDetect = time.time()-timeOfLastFaceDetect\n   \n    if timeSinceLastDetect > timeAllowedWithNoFaceDetectBeforeWarning:\n         PlayWarningIfNeeded()\n        \n    text = \"Seconds since face: {:.1f} \".format(timeSinceLastDetect)\n    frame = cv2.putText(frame, text, (10,dispH-65),fnt, 1.5,(0,0,255), 2)\n\n    text = \"Warnings: {} \".format(warningCount)\n    frame = cv2.putText(frame, text, (10,dispH-120),fnt, 1.5,(0,255,255), 2)\n\n    cv2.imshow('nanoCam',frame)\n    if cv2.waitKey(10)==ord('q') or cv2.getWindowProperty('nanoCam',1) &lt; 1:\n        bCloseProgram = True\n    \n    if (bCloseProgram):\n        break\n\ncam.release()\ncv2.destroyAllWindows()<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Imagine you HAVE to sleep on your back for some reason and possibly restrict neck movement during the night. Here are some options: Tennis balls strapped to sides Placing an iphone on chest\/pocket and using an app (SomnoPose) that monitors position with the accelerometer and beeps when it detects angle changes. (it works ok but [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,32,6],"tags":[],"class_list":["post-2633","post","type-post","status-publish","format-standard","hentry","category-development","category-jetson-nano","category-tech-tips"],"_links":{"self":[{"href":"https:\/\/www.codedojo.com\/index.php?rest_route=\/wp\/v2\/posts\/2633","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.codedojo.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.codedojo.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.codedojo.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.codedojo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2633"}],"version-history":[{"count":7,"href":"https:\/\/www.codedojo.com\/index.php?rest_route=\/wp\/v2\/posts\/2633\/revisions"}],"predecessor-version":[{"id":2640,"href":"https:\/\/www.codedojo.com\/index.php?rest_route=\/wp\/v2\/posts\/2633\/revisions\/2640"}],"wp:attachment":[{"href":"https:\/\/www.codedojo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2633"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.codedojo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2633"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.codedojo.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2633"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}