VR for Industrial Designers with LEAP Motion

A. Background

V2B stands for VR 2 Biz. It is a term borrowed from the B2B and B2C which are biz 2 biz and biz to consumer. Many of the VR products right now is B2C that individuals play games with it. Also, in terms of creating new objects in the VR, it is B2C products that people love to sketch and enjoy the experience of doodling. The picture below depicts the user case of the drawing in the VR.

In terms of B2B, the VR/AR is combining hand gesture to the VR/AR experience. Another user case would be an auto industry which creates an interior of the car and experience the passenger environment in the VR. The picture below shows the user using a hand to modify 3d object.

B. Accuracy

However, we need an accuracy in terms of modification.  We don’t want a dimension by eyeballing hand gestures. We need something like Autocad that has multiple digit accuracy.

In order to achieve the accuracy with hand gestures, I take a look at the micrometer. Micrometer has a dial system, a thimble and a sleeve. The thimble and sleeve  are assigned to two different scales. Sleeve is set to 0.025 inch and the thimble is set to 0.001inch.  Taking this system, I assigned each finger with different scales to achieve the accuracy.  

Here are the fingers that are assigned to the different scales.

C. 3 Axes – colors

Since the accuracy is achievable, now it needs the three dimensions. Precisely, it needs X, Y, Z. X, Y, Z are color coded with RGB at almost every software. So, XYZ::RGB is set in terms of color. So in the V2B when the axes are changed the light color changes accordingly. 

D. Hand Gestures

Since I am going to use the leap motion and the leap motion gives a distance between fingers, I take a look at the sign languages and made a chart on which distance that I need to consider. I made two versions. One is open-palm starting and the other is fist starting.

E. Feedback.

After presenting to multiple users, people gave me a feedback on the hand gestures. Most of the people understood the 3Axes because it is natural to the engineers. Engineers uses X,Y,Z with three fingers, thumb, index and middle. However, people were confused with the scale. Marek Walczak gave a nice feedback based on his architect background. Using a thumb as positive and pinky as negative. So, I finalized the hand gestures.

F. Development

During the development, I made a wooden stand that I can put the leap motion and the smartphone at the same time. It allowed both hands to work with the leap motion freely also, it allowed me to document the development with the phone camera.

Also during the demo, I found it assigning each fingers with scale creates a hurdle to modify. As a result, I simplified with plus and minus as an intro to the new interface. 

G. V2B.

Here is the final development of the V2B with the oculus and the leap motion.

Pre – AR Test

Making a Truly Synchronous Chat by Matching Word Bubbles

From the simple delivery of the letter by horse-back riding in ancient times, we know that people has demanded the synchronous information delivery between people.

However, as a social norm even if we are in the live chat with video and sound, we know that it is impossible to have the truly synchronous delivery of the information.

The maximum limit of this demand only  reach to the point where one start talking the other person has to stop in order to listen to other person.

So, here I wanted to experiment, what if we talk truly instantly by chatting with matching word bubbles.

Through out the experiment, I found that even if we have this matching system, one user waits until the other user  stops writing. It is just a matter of time human brain processing.

LINK TO THE WORK: https://dyk286.itp.io:8080/index3.html

1. BACKGROUND

We have a common sense in conversation that we keep ourselves quiet when the other person speaks. It is true that overlapping one’s voice over the other is rude. However, we have this social norm implemented in the chatting too. The chatting is synchronous platform and we can read the text as we are typing.  So I want to make synchronous text chatting.I wanted to make a new way of chatting that people keep talking and overlap their voices over the other.

2. DEVELOPMENT

I was introduced with the video and the audio which changed the thought process. Also in terms of data ownership the p2p ID generation is very clear, it is data-free(no data stored).

So, I worked on the css, html, javascript to make this project working. I used Tone.js to make sound just like other chatting webpage.  Here is the link it is working only when I run the server.js. As I am working on it I kept googling to get the answers that I want to have. As I am making this chatting, I changed it to refresh every 1minute. In this way it will make the users in the one starting point as time goes. It is the starting time of getting synchronous. So I tested the chatting with people.

3. User Test

First I wanted to have the functional completion. I chat with the friend in France and it proved the P2P is possible World-Wide and during this test I learned the lesson that in terms of video chatting, the video should go with the audio. However, in terms of drunken chat, the audio and video are constraints because it breaks the need of word bubble. So, I muted the audio and shrank the size of the video. I conceptualized it as an on-air kind sign of the broadcasting.

4. Design/Naming

I had a hard time struggling with the design and colors. At the end of the design, I made the design more seamless and made it more crisp by shadow. Also as I design the div I thought of the matching word bubbles as the fistbump. So I made a logo looks like a fist-bump.

5. Modification

I modified the design into something that I like not following Google’s material design because material design does not covey my philosophy, fun, human interactivity.

❒ LETS : Efficient Platform to Make People Social in Real World.

❒ Online services promote strengthening the physical community by joining the online community these days. However, after people joining the online community based on the physical relationship, the actual physical interaction weakens over time. We have hundreds of Facebook friends but it only ties strongly in online but not in actual life. So I wanted make an ‘efficient platform’ that quickly completes the user/group interactions with the ‘actual behavior.’

❒ In 2015, September 15 at Boston, there was a premier of the Black Mass. From this event, the one elderly woman got viral over the internet. She became an online hero because even if there was Johnny Depp she was the only person not taking picture. A futurist, Ray Kurzweil predicts digital implant in human body. He questions people’s dependency on smart phones are an example of brain extender. As we are counting on the digital devices we might lost the reality in our lives. Then, what does the digital devices provide for us? Digital devices gave us efficiency in our lives. We use the emojis to express one’s feeling as a shorthand in our text messaging. So we can see that people are looking of the efficiency in lives.

In terms of text messaging when there are multiple people, it is hard to get to the point when we are scheduling something together. One might ask for Friday night, another can’t make it on Friday, and the other is looking for other time. So, I wanted an efficient way to scheduling. Changing the process from “Talk and do co-activity” to “Decide co-activity and talk” will give a different behavior.

At ITP, people enjoy playing foosball. However finding three other players is a painful process. As time goes we made our efficient gestures. As soon as people make an eye contact both fists are shaken. As a reply, people either nodded or shaken their heads. From here, I wanted to make an efficient platform between friends. It involves a User Interface and new language. So, I wanted a simplest language to kickstart an action. This web app quickly disseminates the activities that any user wants to do to every friends. Before developing an app, I found the key how to express human activities; 5W1H. So, I trimmed it to 4, What, Where, When, Who. Each topic will have options. What: Drink, Sports, Food, Attractions. Where: 0 mile, 1 mile, 3 mile, 5mile. When: Now, 1hour later, 2hour later, 3 hour later, 4hour later. Who: 1person, 2people, 3 people, 4 people. As an user/erector, it will quickly click icons and send it to all of the friends and the friends will join the chat. In terms of language I wanted make a new application of emoji to describe a schedule.

Machine Learning will soon mock human brains in every aspects even in anything that deals with human emotions. We know many of the pop music now are computer-optimized. Industrial revolution replaced the human muscles. Now, machine learning is replacing human brains. For example, if we have nothing but autonomous driving cars, would cars need signal lights? If every car is connected, we would not even need traffic lights. In the future, something we had as a tool for communication will not be needed. So, the chatting that I am building is going for the future, you don’t talk, you don’t have to learn same language as your friend. Only thing you need is just understanding of the icons. If you watched Facebook F8 2016, the key is nothing but better expression and better share. Facebook used to have a solution to express and snapchat took over by video. Now facebook is competing against by adding a new features. So, we can see the trend how it is evolving.

Text ⇢ Picture ⇢ Video

So, I am heading to the other side of expression, even before text, the Icon. It is better sharing because they share a moment by doing same activity.

… … …

A. Background
When I was at the NYU ITP, people loved to play foosball but it has been painful to actually play… Gathering 4 people on the foosball was painful that we had go and ask people ‘you wanna play?’So, I wanted a simplest way to kickstart an action. This web app quickly disseminates the activities that an user wants to do to every friends. Before developing an app, I found the key how to express human activities; 5W1H. So, I trimmed it to 4, What, Where, When, Who. As an user/erector, it will quickly click icons and send it to all of the friends and the friends will accept/deny the message.

B. Machine Learning & Utopia
Many people questioned me about the term, ‘Utopia.’ I borrowed it from Kanye West’s interview.
“What’s your version of utopia? I don’t think people are going to talk in the future. They’re going to communicate through eye contact, body language, emojis, signs.    Imagine that. If everyone was forced to learn sign language. ”

Kanye West: Free Form

The reason why I liked the term ‘Utopia’ was because it is the future we are heading. Machine Learning will soon mock human brains in every aspects even in anything that deals with human emotions. We know many of the pop music now are computer-optimized. Industrial revolution replaced the human muscles. Now, machine learning are replacing human brains. For example, if we have nothing but autonomous driving cars, would cars need signal lights? If every car is connected, we would not even need traffic lights. In the future, something we had as a tool for communication will not be needed. So, the chatting that I am building is going for the future, you don’t talk, you don’t have to learn same language as your friend. Only thing you need is just understanding the icons.

C. Exclusive VS Inclusive
If you look at the facebook, snapchat, instagram they all have an exclusiveness. You can limit your post to your friends excluding people. It is because of facebook which started for the people who have emails ending with “.edu.” Lets that I am creating is inclusive to every friends on the list because it is action oriented not friendliness-oriented. Action oriented means instead of being behind the computer, it makes people to get together and do the activities together.

D. Expression & Share
If you watched Facebook F8 2016, the key is nothing but better expression and better share. Facebook used to have a solution to express well and snapchat took over by video. Now facebook is using instagram by adding a new feature story. So, we can see the trend how it is evolving. Text>Picture>Video. So, I am heading to the other side of expression, even before text, the Icon.

Icon ⇠   ◉   Text ⇢ Picture ⇢ Video

Instead of competing with other platforms, I am targeting the other side of expression. It is better sharing because they share a moment by doing same activity. We don’t need VR to mock being together. https://youtu.be/ouE6qyTc-l0?t=8m43s

E. Google VS Instagram
When you search do you lie what you need? When you post picture on the instagram do you post anything ugly? The data Google is collecting is pure data. People do not lie what they are searching for. On the other hand, data that instagram collects are not pure because people on the instagram want to post something that can trigger more like buttons. The data Lets will collect will be pure and sincere because you don’t lie to your friends especially when you are scheduling something together. So the future community that lets is going for is Utopian that people do not lie but are honest.

Original Design

1. Proposal
   The original plan was to make a visualization of mathematical equation. If you are familiar with Maya there is phong function. The function that changes the surface of the object in 3D. However, the Phong is named after a mathematician whose name is Phong. So, what we see on the screen is often from the mathematical equation done by some mathematicians. So, my goal was to make a visual representation of the mathematical equation.
2. Average Equation
   So, I started with a simple equation using array. The reason that I thought of using array is that because the value that influence the visual got to be changing over time. So, I started with average equation. It was easy approach because the array can be divided by the length of the array. So, mouse X values are summed up in the array and the sum of mouseX is divided by the length of the array. I divided the windowwidth into two that if the mouse X is on the left side, it will be the negative value and if the mouse X is on the right side it would be positive value. In this way the visual would be able to increase and decrease its shape over time.
3. User Testing A
   I showed this to the class and classmates were perplexed by the complexity. They did not get the what the mouseX is doing and why the visual is keep changing. I needed to explain what the logic is behind and clarify the user interaction.
4. User Testing  B
   I simplified the visual and showed it to one of the ITPers and I got a suggestion of using a webcam. So, I created the webcam and mouseX interaction. However, even after, the mouse X was not responding fast enough. I needed something that the value changes dramatically. If you put your mouse on the right side for a long time then, you would be able to see the lines moving and moving objects over the webcam will change the movements of the lines. Also if you put your mouse on the left side for a long time then, you would be able to see the lines are diminishing. https://www.doyoki.com/project/icm/finalfin3/
5. Thanksgiving Break
   I was in the bed, dreamed that I was in the darkness and saw black ellipses moving. So, I woke up and I just opened up the laptop and started to make a black ellipse changing its radius. However, the ellipse was only getting big but small. So, I changed the if statement adding “c = -c;” Now, ellipse was changing big and small.
6. Exponential Equation
   So, I played more with ellipse making more like my dream. I needed an exponential equation. So, I changed the code from  “c = c d” to “c = c d*f.” The “f” is the counter in the for loop so it would change the value in more dramatic as:
   1 1 * 1 = 2
   2 2 * 2 = 6
   6 6 * 3 = 24
   And after changing the equation, when I looked at this sphere getting big and small, I felt like I was in the space approaching to and flying away from the star.
   
7. Final
   I added more colors, more ellipses later. And I eliminated the mouse and the webcam interaction because I wanted to eliminate user’s misunderstandings. Here is the final.
   https://www.doyoki.com/project/icm/finalfin2/
   And here is the visual interpretation of code. You can see the radius of the ellipses are changing in bigger scale.
   https://www.doyoki.com/project/icm/finalfin/
   And here is the code.
   https://gist.github.com/kdoodoo/45074466d268ed84d66a
8. Conclusion
   ICM got me a better understanding of the code. I was in the computer engineering major for the first two years of college and back then I enjoyed elective courses like design 101 more rather than doing the labs on C# and  C . ICM got me better that I like coding now. And I cannot believe there are only three semesters left at the ITP and it feels much different from the first semester of the 4 years of college. I am running out of time. ITP gotta be 4 year pr0gram.

The graphic is from the WEBGL of P5.JS. I made a music composed with FL Studio which is YES. The other is Grace’s recent music, You don’t own me. So, YES conveys my thought process well.

The music is continuously in loop. The musical sense is in a normal state that audience enjoys the music. Once the user is curious about the interaction, it might click. As the user click the mouse button, there is a transition to other view.

If that view is in temptation one will continuously click to explore as the the other music interferes the sense of normality.

And if one keeps clicking on the mouse the music will trans into abnormal state which will lose the sense of the music but it will be a noise.

As a result, I named the work, Anxiety Explosive.

Technical Note: I used the Tone.js, P5.js and WEBGL. The boxes are outcome of analyzer of the tone.js that the box’s XYZ dimensions are stretching and contracting and also it rotates related to the analyzer. Also, when the mouse is pressed, the camera is triggered related to mouse X and Y positions.

FULL SCREEN1

FULL SCREEN2

Human Muscle interpretation in robotic arm mechanism.

Motorized device has a limited mimicry on human/animal muscle movement. Being precise to satisfy industrial demands has allowed robots to replace human labor.

However, it lost a sense of human mimicry in natural movement. Just like musicians avoid exact tempo of the instruments at the MIDI software to make more humane sound,  the movement of the robot hand mocks the human natural motion.

Implementing humanness on the robotic hand is based on the long time ago animal experiment. From this well known experiment on the dead body of the frog,  we can see animal muscle is triggered by electrons.

Making frogs’ leg muscle popping by connecting electricity clearly shows disgust of the combination of the nature and the artificial implant. However, from the scientific sense of  the experiment, it allowed me to make this human-like robot arm.

Instead of digital input to analog output, here both are analog reading and analog output. Magnet was used to convey natural human instability in human motion. Instability means imperfection which is a contrary to industrial robots with G-code.

As one can see, the servo motor is shaking as the magnetic flux sensor is approaching to the magnet.  Just like when human limbs are responding to the objects ahead of touching them directly.

The exploration of this human mimicry will lower the barrier of the denial against artificial robots.

In terms of an assembly, I made pockets bit smaller which is better than bigger. Getting all 4 pieces into the 4 holes was painful. I sanded them and it was now loose by hair. It was fit but easy to remove too. So I glued the top 4 pockets. Here is the result.

The pieces came out nicely except the black laminated piece was painful to get it out. The laminated was not cut clean I had to cut manually. After Cutting manually, I sanded the edges which torn off some of the laminates. So the parts which got torn off are placed at the inside of the table.

At the master cam, I made three different G code because I had three different pieces of woods. Having multiple woods created multiple G codes. However at the CNC, it was much easier because I basically set the woods all together at the bed of the CNC. As CNC finishes one job, I switched the XYZ zeros to the new board and started cutting with the different files.

Here you may see the black wood is aligned next to the second piece.

I grabbed the material that is thick. I looked for the thickness first which later gave me a hard time.

Since the thickness of the aluminum rod was too thick, I changed the teeth of the lathe to the other form. Getting them back in the right position was hard that one at the bottom was loose due to the gravity. So, I tightened them with my hand as screwing the chuck key. And in order to check the safety I tightened them to the maximum.

After the teeth were fit, I placed the rod and started facing. Since the rod was cut not in flat I faced it. It took a lot of oiling, stop and check.

As I face the surface I noticed that the rod was not cutting flat on the outer edge. So I faced from the inside to the outside instead of from the outside to the inside. And the result was good.   

After the facing, I did turning and marked the line that I could turn as I am turning to keep myself safe.

After turning, I marked the design that I wanted to make and I used parting tool and turing tool to making it. After finishing the turning, I drilled the hole at the center. First I used the center bit to mark the center. I used 4 drill bits to get the right size from the chart.

After threading, I wanted to cut the piece into half. I used parting tool to cut then, at the middle of process I was little scared. The piece that I was about to cut was heavy and I was not confident whether the WD40 plastic pipe would catch it. So, I stopped the lathe and cut the other end with the metal saw. The piece was too small to fit in the metal saw. So I tried to cut just 1/4 of the piece. And as I finished cutting, the other piece flew to the plastic container. That was really a scary moment. So later, I would just use the lathe facing the piece to shorten it. So I finished the piece and made the thread with tapping tool.

After finishing, I found that I actually made a hole through as I am making the tap. Here is the result.

After that, I managed the process at the Mastercam. At the Autocad I used different colors to manage the pocketing and the contouring and I noticed the yellow is not readable at the Mastercam because selection made in the Mastercam is yellow.

After generating the G-code, I brought the file into the CNC and it was cutting well.

Conclusion. I noticed that in terms of height, there is a lot of difference between the number that I plugged in with the actual size because the pocketing was a cutting, basically tearing off the board a lot.