Yolo net label output from PC to an Arduino? - arduino

I have recently gained interest over this machine learning topic about image classification.
I am in no way a programmer, but I am a farmer who is very interested about it, and detecting the quality of fruits and vegetables is a very tedious and time consuming task, specially if you don't have the money to buy industrial machinery to perform this task at a small-medium scale.
I recently came across this tutorial (had to fix a lot of errors because it is really bad written, but it works):
https://imaginghub.com/projects/148-how-to-distinguish-apples-and-pears-with-raspberry-pi/documentation
Which basically is the building block of a future fruit/vegetable quality grader.
This conveyor belt is going to have an Arduino that will receive an output from the Python program, that output should activate servos to redirect each fruit/vegetable to its own basket.
Now I would like to know how can I get the label output from the net and transform it to a number for example:
apple = 1, orange = 2, cucumber = 3...
So whenever it's apple, Arduino receives a 1 that will light an LED (first this, then servo), same happens for orange and cucumber and so on.
Here are the 2 codes I believe have to do something with the box label output Deploy.py and yolo_net.py
Deploy.py is the one I run to get the live camera detector:
Deploy.ipynb
And this is yolo_net.py (if it helps somehow to answer my question):
yolo_net.py

All you have to do is modify the yolo_net.py file and go under def draw_detections(self, image, boxes): over to line 63,
it should say
cv2.putText (image, class_name, (left, top - 12), 0, 1e-3 * h, self.colors[class_indx], thick//2)
it is under that line that you should add what you want the arduino to perform.

Related

what is an efficient method for transposing N timing arrays into one master array?

This is an extremely niche problem, so I'll do my best to explain it in words:
Say you have an operation that requires negligible time (in my case, stepping a stepper motor once by pulsing the pins). I want to coordinate the movement of 6 individual motors with their own acceleration curve, but actuate them with the same micro controller. I also want the individual acceleration curves of the motors to be modifiable.
I'm using a Teensy 4.1, so this program will be written in Arduino language (near identical to C++).
My current approach to this problem is to generate six individual "delay" arrays for each motor. Essentially, their speed is controlled by the delay in between each pulse, and the angular distance traveled by the # of steps, or elements of each delay array. Something like this:
1 (P20) 1 (P20) 1 (P20) 1
2 (P30) 2 (P30) 2 (P30) 2
Where a 1 or 2 is the respective motor's step and a (PX) is a delay for X seconds.
I would like to write some master transposition function that turns the above into this:
1,2 (P20) 1 (P10) 2 (P10) 1,2 (P20) 1 (P10) 2
This array, when read by my actuation code would step motors 1 and 2 at the same time, wait 20 microseconds, step motor 1, 10 microseconds, step motor 2... etc.
It seems pretty simple when you do it for two motors, but for some reason I just can't wrap my head around making a completely modular version of this. In my case, it would need to merge 6 arrays into one.
I am also just wondering if anyone could think of a more elegant solution to this problem, as I am pretty new to programming and don't know all the features / capabilities of C++.
I have tried applying an iterative method, where you keep track of the current total delay elapsed and subtract it from the lowest next total delay of the different stepper arrays, then append the master array with that difference and update all the totals accordingly, but this approach always ends up too convoluted for me to follow.
Might I offer a different solution:
Use something like freeRTOS to and make individual threads to control each motor. That way you can define delays for each motor individually without having the extra complication of making a timing array.
The ESP32 has freeRTOS available in the Arduino definitions and is pretty easy to use. You could find a port for the Teensy 4.1
Here is a really nice tutorial for blinking LEDs at different rates. Your code should have a similar approach.
freeRTOS ESP32 tutorial

Trying to interface with an Aanderaa RCM9

I have an older Aanderaa RCM9 (https://epic.awi.de/id/eprint/45145/1/RCM9.pdf) that is missing its Data Storage Unit and its reader. They don't produce these anymore nor do they service the model. It would be a shame to toss an otherwise nice piece of equipment, so I thought to try and get a serial feed from the terminal or DSU output and log on an Arduino with an SD card. I have tried to connect with a TTL-RS232 converter, and there seems to be a consistent Tx from the instrument, it comes in batches, but reads out in CoolTerm as "............" I've tried different terminal configurations, and connections, but that's the best I get. Here's how it looks inside: https://imgur.com/a/xxCPUlQ
Any thoughts??
I am afraid that the output is the old Aanderaa PDC4 serial format where long pulses (81ms) represents zeros and short (27ms) represents ones in a 10 bit binary word framed in a 4 second window.

R read data from a txt space delimited file with quoted text

I'm trying to load a dataset into R Studio, where the dataset itself is space-delimited, but it also contains spaces in quoted text like in csv files. Here is the head of the data:
DOC_ID LABEL RATING VERIFIED_PURCHASE PRODUCT_CATEGORY PRODUCT_ID PRODUCT_TITLE REVIEW_TITLE REVIEW_TEXT
1 __label1__ 4 N PC B00008NG7N "Targus PAUK10U Ultra Mini USB Keypad, Black" useful "When least you think so, this product will save the day. Just keep it around just in case you need it for something."
2 __label1__ 4 Y Wireless B00LH0Y3NM Note 3 Battery : Stalion Strength Replacement 3200mAh Li-Ion Battery for Samsung Galaxy Note 3 [24-Month Warranty] with NFC Chip + Google Wallet Capable New era for batteries Lithium batteries are something new introduced in the market there average developing cost is relatively high but Stallion doesn't compromise on quality and provides us with the best at a low cost.<br />There are so many in built technical assistants that act like a sensor in their particular forté. The battery keeps my phone charged up and it works at every voltage and a high voltage is never risked.
3 __label1__ 3 N Baby B000I5UZ1Q "Fisher-Price Papasan Cradle Swing, Starlight" doesn't swing very well. "I purchased this swing for my baby. She is 6 months now and has pretty much out grown it. It is very loud and doesn't swing very well. It is beautiful though. I love the colors and it has a lot of settings, but I don't think it was worth the money."
4 __label1__ 4 N Office Products B003822IRA Casio MS-80B Standard Function Desktop Calculator Great computing! I was looking for an inexpensive desk calcolatur and here it is. It works and does everything I need. Only issue is that it tilts slightly to one side so when I hit any keys it rocks a little bit. Not a big deal.
5 __label1__ 4 N Beauty B00PWSAXAM Shine Whitening - Zero Peroxide Teeth Whitening System - No Sensitivity Only use twice a week "I only use it twice a week and the results are great. I have used other teeth whitening solutions and most of them, for the same results I would have to use it at least three times a week. Will keep using this because of the potency of the solution and also the technique of the trays, it keeps everything in my teeth, in my mouth."
6 __label1__ 3 N Health & Personal Care B00686HNUK Tobacco Pipe Stand - Fold-away Portable - Light Weight - For Single Pipe not sure I'm not sure what this is supposed to be but I would recommend that you do a little more research into the culture of using pipes if you plan on giving this as a gift or using it yourself.
7 __label1__ 4 N Toys B00NUG865W ESPN 2-Piece Table Tennis PING PONG TABLE GREAT FOR YOUTHS AND FAMILY "Pleased with ping pong table. 11 year old and 13 year old having a blast, plus lots of family entertainment too. Plus better than kids sitting on video games all day. A friend put it together. I do believe that was a challenge, but nothing they could not handle"
8 __label1__ 4 Y Beauty B00QUL8VX6 "Abundant Health 25% Vitamin C Serum with Vitamin E and Hyaluronic Acid for Youthful Looking Skin, 1 fl. oz." Great vitamin C serum "Great vitamin C serum... I really like the oil feeling, not too sticky. I used it last week on some of my recent bug bites and it helps heal the skin faster than normal."
9 __label1__ 4 N Health & Personal Care B004YHKVCM PODS Spring Meadow HE Turbo Laundry Detergent Pacs 77-load Tub wonderful detergent. "I've used tide pods laundry detergent for many years,its such a great detergent to use having a nice scent and leaver the cloths smelling fresh."
Problem is that it looks tab-delimited but it is not, example would be DOC_ID = 1, where there are only two spaces between useful and "When least...", this way passing sep = "/t" to read.table throws an error saying that line 1 did not have 10 elements, which for some reason is incorrect, because the number of elements should be 9. Here are the parameters that I'm passing(without the original path):
read.table(file = "path", sep ="\t", header = TRUE, strip.white = TRUE)
Also relying on quotes is not a good strategy, because some lines do not have their text quoted, so the delimiter should be something like a double space, which combined with strip.white should work properly, but read.table only accepts single byte delimiters.
So the question is how would you parse such corpus in R or with any other third party software that could convert it adequately to a csv or atleast a tab-delimited file?
Parsing the data using python pandas.read_csv(filename, sep='\t', header = 0, ...) seems to have parsed the data successfully and from this point anything could be done with it. Closing this out.

Problem on QT GUI Frequency Sink GNU Radio Windows

I tried to generate a Gaussian noise signal having a standard deviation of 1 by setting the amplitude value in the noise source configuration to 1.
When I tried to display it using the QT GUI Frequency Sink, I initially expected that the resulting PSD would fluctuate around 0 dB across all frequency points. However, I have found that the PSD displayed at QT GUI Frequency Sink fluctuates around -40 dB (instead of 0 dB) over all frequency points.
From the signal processing theory's point of view, this result is clearly incorrect.
Is there a bug in my app that I don't know?
I am using GNU Radio application v3.8.2.0-57-gd71cd177 (Python 3.9.0) on windows 10.
Two things: there's a bug in your expectation:
the sum of all powers needs to be around 1, i.e. 0dB, because that's the power and thus the variance, so the individual bins need to be lower - in fact, the lower the longer your FFT is.
But there's also undocumented scaling in the frequency sink, so we can call that a GNU Radio bug, if we want. GNU Radio is aware of the problem. It's not easy to address because it's bad to have unexpected behaviour, but it's also bad to break existing applications.

Showing too much 'skin' detection in software

I am building an ASP.NET web site where the users may upload photos of themselves. There could be thousands of photos uploaded every day. One thing my boss has asked a few time is if there is any way we could detect if any of the photos are showing too much 'skin' and automatically move flag these as 'Adults Only' before the editors make the final decision.
Your best bet is to deal with the image in the HSV colour space (see here for rgb - hsv conversion). The colour of skin is pretty much the same between all races, its just the saturation that changes. By dealing with the image in HSV you can simply search for the colour of skin.
You might do this by simply counting the number of pixel within a colour range, or you could perform region growing around pixel to calculate the size of the areas the colour.
Edit: for dealing with grainy images, you might want to perform a median filter on the image first, and then reduce the number of colours to segment the image first, you will have to play around with the settings on a large set of pre-classifed (adult or not) images and see how the values behave to get a satisfactory level of detection.
EDIT: Heres some code that should do a simple count (not tested it, its a quick mashup of some code from here and rgb to hsl here)
Bitmap b = new Bitmap(_image);
BitmapData bData = b.LockBits(new Rectangle(0, 0, _image.Width, _image.Height), ImageLockMode.ReadWrite, b.PixelFormat);
byte bitsPerPixel = GetBitsPerPixel(bData.PixelFormat);
byte* scan0 = (byte*)bData.Scan0.ToPointer();
int count;
for (int i = 0; i < bData.Height; ++i)
{
for (int j = 0; j < bData.Width; ++j)
{
byte* data = scan0 + i * bData.Stride + j * bitsPerPixel / 8;
byte r = data[2];
byte g = data[1];
byte b = data[0];
byte max = (byte)Math.Max(r, Math.Max(g, b));
byte min = (byte)Math.Min(r, Math.Min(g, b));
int h;
if(max == min)
h = 0;
else if(r > g && r > b)
h = (60 * ((g - b) / (max - min))) % 360;
else if (g > r && g > b)
h = 60 * ((b - r)/max - min) + 120;
else if (b > r && b > g)
h = 60 * ((r - g) / max - min) + 240;
if(h > _lowerThresh && h < _upperThresh)
count++;
}
}
b.UnlockBits(bData);
Of course, this will fail for the first user who posts a close-up of someone's face (or hand, or foot, or whatnot). Ultimately, all these forms of automated censorship will fail until there's a real paradigm-shift in the way computers do object recognition.
I'm not saying that you shouldn't attempt it nontheless; but I want to point to these problems. Do not expect a perfect (or even good) solution. It doesn't exist.
I doubt that there exists any off-the-shelf software that can determine if the user uploads a naughty picture. Your best bet is to let users flag images as 'Adults Only' with a button next to the picture. (Clarification: I mean users other than the one who uploaded the picture--similar to how posts can be marked offensive here on StackOverflow.)
Also, consider this review of an attempt to do the same thing in a dedicated product: http://www.dansdata.com/pornsweeper.htm.
Link stolen from today's StackOverflow podcast, of course :).
We can't even write filters that detect dirty words accurately in blog posts, and your boss is asking for a porno detector? CLBUTTIC!
I would say your answer lies in crowdsourcing the task. This almost always works and tends to scale very well.
It doesn't have to involve making some users into "admins" and coming up with different permissions - it can be as simple as to enable an "inappropriate" link near each image and keeping a count.
See the seminal paper "Finding Naked People" by Fleck/Forsyth published in ECCV. (Advanced).
http://www.cs.hmc.edu/~fleck/naked.html
Interesting question from a theoretical / algorithmic standppoint. One approach to the problem would be to flag images that contain large skin-colored regions (as explained by Trull).
However, the amount of skin shown is not a determinant of an offesive image, it's rather the location of the skin shown. Perhaps you can use face detection (search for algorithms) to refine the results -- determine how large the skin regions are relative to the face, and if they belong to the face (perhaps how far below it they are).
I know either Flickr or Picasa has implemented this. I believe the routine was called FleshFinder.
A tip on the architecture of doing this:
Run this as a windows service separate from the ASP.NET Pipeline, instead of analyzing images in real time, create a queue of new images that are uploaded for the service to work through.
You can use the normal System.Drawing stuff if you want, but if you really need to process a lot of images, it would be better to use native code and a high performance graphics library and P/invoke the routine from your service.
As resources are available, process images in the background and flag ones that are suspicious for editors review, this should prune down the number of images to review significantly, while not annoying people who upload pictures of skin colored houses.
I would approach the problem from a statistical standpoint. Get a bunch of pictures that you consider safe, and a bunch that you don't (that will make for a fun day of research), and see what they have in common. Analyze them all for color range and saturation to see if you can pick out characteristics that all of the naughty photos, and few of the safe ones have.
Perhaps the Porn Breath Test would be helpful - as reported on Slashdot.
Rigan Ap-apid presented a paper at WorldComp '08 on just this problem space. The paper is allegedly here, but the server was timing out for me. I attended the presentation of the paper and he covered comparable systems and their effectiveness as well as his own approach. You might contact him directly.
I'm afraid I can't help point you in the right direction, but I do remember reading about this being done before. It was in the context of people complaining about baby pictures being caught and flagged mistakenly. If nothing else, I can give you the hope that you don't have to invent the wheel all by yourself... Someone else has been down this road!
CrowdSifter by Dolores Labs might do the trick for you. I read their blog all the time as they seem to love statistics and crowdsourcing and like to talk about it. They use amazon's mechanical turk for a lot of their processing and know how to process the results to get the right answers out of things. Check out their blog at the very least to see some cool statistical experiments.
As mentioned above by Bill (and Craig's google quote) statistical methods can be highly effective.
Two approaches you might want to look into are:
Neural Networks
Multi Variate Analysis (MVA)
The MVA approach would be to get a "representative sample" of acceptable pictures and of unacceptable pictures. The X data would be an array of bytes from each picture, the Y would be assigned by you as a 1 for unacceptable and a 0 for acceptable. Create a PLS model using this data. Run new data against the model and see how well it predicts the Y.
Rather than this binary approach you could have multiple Y's (e.g. 0=acceptable, 1=swimsuit/underwear, 2=pornographic)
To build the model you can look at open source software or there are a number of commercial packages available (although they are typically not cheap)
Because even the best statistical approaches are not perfect the idea of also including user feedback would probably be a good idea.
Good luck (and worst case you get to spend time collecting naughty pictures as an approved and paid activity!)

Resources