{"id":1118,"date":"2023-02-13T09:09:35","date_gmt":"2023-02-13T01:09:35","guid":{"rendered":"https:\/\/pixeling.net\/work\/wp\/?post_type=rara-portfolio&#038;p=1118"},"modified":"2026-07-09T07:30:40","modified_gmt":"2026-07-08T23:30:40","slug":"skeletal-keypoints","status":"publish","type":"rara-portfolio","link":"https:\/\/pixeling.net\/work\/wp\/?rara-portfolio=skeletal-keypoints","title":{"rendered":"Skeletal Key Point"},"content":{"rendered":"<h4>Overview<\/h4>\n<p>Data annotation involves labeling or tagging data to make it comprehensible and usable for Artificial Intelligence and machine learning algorithms.<\/p>\n<p><span style=\"font-weight: 400;\">Skeletal Key Point was one of the features in the Video Annotation (VA) tool which allowed users to label human poses or anatomy of an object which can then be used to train the algorithms.\u00a0<\/span><\/p>\n<h4>Problem Statement<\/h4>\n<p><span style=\"font-weight: 400;\">One of our clients requested this feature as they received inaccurate data results that were sent to us. <\/span><span style=\"font-weight: 400;\">Users were unable to annotate accurately with the existing Bounding Box tool, thus the data results were not detailed enough as it had some missing information about human poses.<\/span><\/p>\n<h4>Goal<\/h4>\n<p><span style=\"font-weight: 400;\">Therefore, the main goal of the Skeletal Key Point project was to solve one of our existing client&#8217;s issues with their video annotation work. They wanted to use our platform and expertise to import data and annotate human poses with higher quality and accuracy<\/span>.<\/p>\n<p><span style=\"font-weight: 400;\">It achieved this by providing a visualisation of the functional connections between key points, which was one of the basic annotation tools in data labelling. <\/span><span style=\"font-weight: 400;\">The connections made annotation errors very noticeable, and therefore less costly to fix. This capability was also supported in the review portal which allowed clients to review the annotated work, and it made the quality review and acceptance easier and less tedious.<\/span><\/p>\n<h4>Users &amp; Audience<\/h4>\n<p><span style=\"font-weight: 400;\">There were 2 personas who used this feature:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The <strong>administrator<\/strong> who configures and enable the tool in the system (Control Portal).<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The <strong>worker<\/strong> who used the tool (VA tool) to localize the skeletal key points.<\/span><\/li>\n<\/ol>\n<p><iframe loading=\"lazy\" style=\"border: 1px solid rgba(0, 0, 0, 0.1); margin-top: 20px;\" src=\"https:\/\/www.figma.com\/embed?embed_host=share&amp;url=https%3A%2F%2Fwww.figma.com%2Ffile%2F6eYN6uj8zQXnb2QnmuCvl6%2FSkeletal-Keypoints%3Ftype%3Ddesign%26node-id%3D208%253A55332%26t%3DVAAsC15QsdHtSpsX-1\" width=\"800\" height=\"450\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<h4><\/h4>\n<h4><\/h4>\n<h4>Roles &amp; Responsibilities<\/h4>\n<p><span style=\"font-weight: 400;\">I was the designer for this project, working closely with the Product Owners and developers in the US and KL teams. <\/span><span style=\"font-weight: 400;\">The US team was responsible for the VA tool, while the KL team was responsible for the configurations in the Control Portal.<\/span><\/p>\n<h4>Scope &amp; Constraints<\/h4>\n<p><b style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif;\">Limited information and domain knowledge<\/b><\/p>\n<p><span style=\"font-weight: 400;\">I have to admit that I am not familiar with most of the technology in the AI industry, and being assigned to this project sounded really daunting.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">There were not many pose estimation tools that I could refer to, and I found that most of the annotation tools offered the same method of annotating. They required the annotators to build a template in the configuration system, and this template will be imported into the annotation tool. The annotators will have to move the key points one by one to adjust the template based on the object in the video or image. This makes it very time-consuming and tedious.<\/span><\/p>\n<p><b>Limitations of technology in the existing system<\/b><\/p>\n<p><span style=\"font-weight: 400;\">As I was not familiar with the technology, I had to decide whether to follow the existing tools which are available in the market, or to make use of the existing features in our system. Building a new set of features just to support the Skeletal Key Point capability was very costly and we had a tight deadline to meet. I had a discussion with the Lead Developer and Product Owner and we decided to make use of the hierarchy relationship feature in our system to build the Skeletal Key Point.<\/span><\/p>\n<h4><\/h4>\n<h4>The Process and Solution<\/h4>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-1377 size-large\" src=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/sk-320-1024x480.jpg\" alt=\"\" width=\"750\" height=\"352\" srcset=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/sk-320-1024x480.jpg 1024w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/sk-320-300x141.jpg 300w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/sk-320-768x360.jpg 768w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/sk-320-128x60.jpg 128w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/sk-320.jpg 1106w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">I came up with a user flow based on my research on the existing tools from competitors and also interviewing the annotators who used the VA tool, as I felt that these two methods would be the fastest\u00a0 in getting information about the VA tool.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Based on the flow, there are two personas involved; the Admin who sets up the skeletal key point configuration in the Control platform, and the Worker, who localizes the skeletal key point in the Worker Portal platform.<\/span><\/p>\n<p><iframe loading=\"lazy\" style=\"border: 1px solid rgba(0, 0, 0, 0.1);\" src=\"https:\/\/www.figma.com\/embed?embed_host=share&amp;url=https%3A%2F%2Fwww.figma.com%2Ffile%2F48uVbGeKVRheVkRwenCBsi%2FVA5-Design-Spec---Primary%3Fnode-id%3D207%253A123175%26t%3DeLUSI8hs8UE3U8Cj-1\" width=\"800\" height=\"450\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p><span style=\"font-weight: 400;\">I&#8217;m going to focus more on the worker side, as this involved a lot of interactions on the tool. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Therefore, instead of having the user create a template in the system and importing the template into the annotation tool based on the tools available in the market, we decided to make use of our hierarchical feature in our system. Based on the workflow in the Figma file, this was what I came up with:<\/span><\/p>\n<p><iframe loading=\"lazy\" style=\"border: 1px solid rgba(0, 0, 0, 0.1);\" src=\"https:\/\/www.figma.com\/embed?embed_host=share&amp;url=https%3A%2F%2Fwww.figma.com%2Ffile%2F48uVbGeKVRheVkRwenCBsi%2FVA5-Design-Spec---Primary%3Fnode-id%3D19%253A123175%26t%3DYstRuJvRR7lIZTOh-1\" width=\"800\" height=\"450\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p><a href=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-1324\" src=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal.jpg\" alt=\"\" width=\"1761\" height=\"1017\" srcset=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal.jpg 1761w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal-300x173.jpg 300w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal-1024x591.jpg 1024w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal-768x444.jpg 768w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal-330x190.jpg 330w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal-1536x887.jpg 1536w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/proto-skeletal-104x60.jpg 104w\" sizes=\"auto, (max-width: 1761px) 100vw, 1761px\" \/><\/a><\/p>\n<p><span style=\"font-weight: 400;\">The user can annotate right away in the annotation tool by localizing the key points in the canvas. During the localization process, the key points will be connected automatically with a \u2018skeletal line\u2019. Once it is completed, a \u2018skeletal frame\u2019 is rendered.<\/span><\/p>\n<div class=\"wp-playlist wp-video-playlist wp-playlist-light\">\n\t\t<video controls=\"controls\" preload=\"none\" width=\"728\"\n\t\t height=\"410\"\t><\/video>\n\t<div class=\"wp-playlist-next\"><\/div>\n\t<div class=\"wp-playlist-prev\"><\/div>\n\t<noscript>\n\t<ol>\n\t\t<li><a href='https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeletal-keypoints.mp4'>Annotating skeletal key points<\/a><\/li>\t<\/ol>\n\t<\/noscript>\n\t<script type=\"application\/json\" class=\"wp-playlist-script\">{\"type\":\"video\",\"tracklist\":false,\"tracknumbers\":true,\"images\":true,\"artists\":true,\"tracks\":[{\"src\":\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeletal-keypoints.mp4\",\"type\":\"video\/mp4\",\"title\":\"Annotating skeletal key points\",\"caption\":\"\",\"description\":\"\",\"meta\":{\"length_formatted\":\"2:43\"},\"dimensions\":{\"original\":{\"width\":1920,\"height\":1080},\"resized\":{\"width\":728,\"height\":410}},\"image\":{\"src\":\"https:\/\/pixeling.net\/work\/wp\/wp-includes\/images\/media\/video.svg\",\"width\":48,\"height\":64},\"thumb\":{\"src\":\"https:\/\/pixeling.net\/work\/wp\/wp-includes\/images\/media\/video.svg\",\"width\":48,\"height\":64}}]}<\/script>\n<\/div>\n\t\n<p><span style=\"font-weight: 400;\">The \u2018skeletal line\u2019 is configured in the system by using the existing hierarchical relationship feature.<\/span><\/p>\n<div id='gallery-1' class='gallery galleryid-1118 gallery-columns-1 gallery-size-large'><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"750\" height=\"469\" src=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton4-1024x640.jpg\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton4-1024x640.jpg 1024w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton4-300x188.jpg 300w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton4-768x480.jpg 768w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton4-96x60.jpg 96w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton4.jpg 1440w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon portrait'>\n\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"750\" height=\"1008\" src=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3-762x1024.jpg\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3-762x1024.jpg 762w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3-223x300.jpg 223w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3-768x1031.jpg 768w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3-1144x1536.jpg 1144w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3-45x60.jpg 45w, https:\/\/pixeling.net\/work\/wp\/wp-content\/uploads\/2023\/02\/skeleton3.jpg 1440w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/>\n\t\t\t<\/div><\/figure>\n\t\t<\/div>\n\n<p><iframe loading=\"lazy\" style=\"border: 1px solid rgba(0, 0, 0, 0.1);\" src=\"https:\/\/www.figma.com\/embed?embed_host=share&amp;url=https%3A%2F%2Fwww.figma.com%2Ffile%2FoIK7HBmYZKfJ67UyGmIeWD%2FCompleted-Projects%3Fnode-id%3D0%253A1%26t%3DIzqfcaVlt5Q1p9J0-1\" width=\"800\" height=\"450\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<h4>Conclusion<\/h4>\n<p><span style=\"font-weight: 400;\">This feature does not only cater to human pose estimation, as it can be used for other types of annotations as well, ranging from annotating inanimate objects, for example, cars, to the anatomy of an animal.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">After the project was launched, the response from the stakeholders was very good. There were plans to improve this feature such as developing the system to be intelligent enough to localize a key point automatically as the annotation moves along the hierarchical list, but for now, I am happy with the result as this was a challenging project to work on. I have learned a lot while working on this project, especially about pose estimation in the AI industry which I was unfamiliar with in the beginning.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview Data annotation involves labeling or tagging data to make it comprehensible and usable for Artificial Intelligence and machine learning algorithms. Skeletal Key Point was one of the features in the Video Annotation (VA) tool which allowed users to label human poses or anatomy of an object which can then be used to train the &hellip; <\/p>\n","protected":false},"author":1,"featured_media":1380,"comment_status":"closed","ping_status":"closed","template":"","rara_portfolio_categories":[2],"class_list":["post-1118","rara-portfolio","type-rara-portfolio","status-publish","has-post-thumbnail","hentry","rara_portfolio_categories-uiux","latest_post"],"_links":{"self":[{"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=\/wp\/v2\/rara-portfolio\/1118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=\/wp\/v2\/rara-portfolio"}],"about":[{"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=\/wp\/v2\/types\/rara-portfolio"}],"author":[{"embeddable":true,"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1118"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=\/wp\/v2\/media\/1380"}],"wp:attachment":[{"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1118"}],"wp:term":[{"taxonomy":"rara_portfolio_categories","embeddable":true,"href":"https:\/\/pixeling.net\/work\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Frara_portfolio_categories&post=1118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}