Progress Report

First I started off with importing the Times Square Subway with almost all the trains, and the 59th street station trains images onto MATLAB.






















This plate displays numerous colors, but in order to make it easier for the camera to capture them, I have to pick up the colors individually.

First I started off with changing the color of the whole image. Using the following MATLAB code, I turned the whole image to gray.

>> rgb = imread('NQR345.png');
>> figure
>> imshow(rgb)
>> gray_image = rgb2gray(rbg);
imshow(gray_image);


After, I was flexible with changes the colors around, I tired to make the change it to black and white. Since there are so many other letter and numbers on the plate itself, we need to detect the area we have to focus on. So using MATLAB, I was able to make the image black and white and able to capture NQR456.

The code:
>> center6 = center(1:6,:);
>> radius  = radii(1:6);
>> metric6 = metric(1:6);
>> viscircles(center6, radii6, 'EdgeColor', 'r');

**The 6 in the code above could be named anything else, for instance, train.

So far I been successful at taking the colors away, now it's time to detect colors. Any colors I want.
MATLAB has an app called Color Thresholding App designed to mix the color or separate the colors. I played around with it using the Times Square Subway image.
The polynomial to the right in the image helps you separate or join the colors. So in the image above, I took away all the colors expect for red and Orange. It even took away the white letters and numbers. We can still see the numbers 1, 2, 3, and letters N, Q, R, and W because initially they were white, and the background is all black.

This is another example of Color Thresholding. In this image I made the polynomial in a way so that we can see all the trains in one look. Although, it;s not completely clear because we can still see some of the white letters,

To take this to the next level, using MATLAB, I took all the colors away, and took away all the numbers and letters away except for 1, 2, and 3. The code to get this answer is the following.

%image = imread('/Users/admin/Pictures/subwaysignall.jpg');

image = imread('subwaysign.jpg');

J = imresize(I, 0.5);

figure

imshow(image)

title('Original Image')

figure

imshow(J)

title('Resized Image')

%123

%image = imread('subwaysignall.jpg');

figure(1), imshow(image), title('Original');

%image = im2double(image);

%[r c p] = size(image);

%imageR = squeeze(image(:,:,1));

%imageG = squeeze(image(:,:,2));

%imageB = squeeze(image(:,:,3));

%imageBWR = im2bw(imageR, graythresh(imageR));

%imageBWG = im2bw(imageG, graythresh(imageG));

%imageBWB = im2bw(imageB, graythresh(imageB));

%imageBW = imcomplement(imageBWR&imageBWG&imageBWB);

%figure(2), imshow(imageBWR); title('Color Thresholded');

%figure(3), imshow(imageBWG); title('Color Thresholded');

%figure(4), imshow(imageBWB); title('Color Thresholded');

%figure(5), imshow(imageBW); title('Color Thresholded');

red = @createMask;

imageBWRed = red(image);

figure(6), imshow(imageBWRed); title('Red Train');

imageBWRed = bwmorph(imageBWRed,'clean')

% ocrAnn = @insertOCRAnnotation;

ocrEval = @evaluateOCRTraining;

[ocrimageBWRed, results] = ocrEval(imageBWRed);

figure(7), imshow(ocrimageBWRed); title('OCR Red Train');

results.Text;

text = results.Text;

words = ['say ' '-v' 'Victoria' ' subway train ' text(1:1) ' ' text(2:2) ' ' text(3:3)];

%system('say -v "Victoria" words');

system(words);

results.CharacterConfidences;

function [BW,maskedRGBImage] = createMask(RGB)

%createMask  Threshold RGB image using auto-generated code from colorThresholder app.

%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using

%  auto-generated code from the colorThresholder App. The colorspace and

%  minimum/maximum values for each channel of the colorspace were set in the

%  App and result in a binary mask BW and a composite image maskedRGBImage,

%  which shows the original RGB image values under the mask BW.

% Auto-generated by colorThresholder app on 03-Dec-2016

%------------------------------------------------------

% Convert RGB image to chosen color space

I = rgb2hsv(RGB);

% Define thresholds for channel 1 based on histogram settings

channel1Min = 0.917;

channel1Max = 0.045;

% Define thresholds for channel 2 based on histogram settings

channel2Min = 0.000;

channel2Max = 1.000;

% Define thresholds for channel 3 based on histogram settings

channel3Min = 0.000;

channel3Max = 1.000;

% Create mask based on chosen histogram thresholds

sliderBW = ( (I(:,:,1) >= channel1Min) | (I(:,:,1) <= channel1Max) ) & ...

    (I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...

    (I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);

% Create mask based on selected regions of interest on point cloud projection

I = double(I);

[m,n,~] = size(I);

polyBW = false([m,n]);

I = reshape(I,[m*n 3]);

% Convert HSV color space to canonical coordinates

Xcoord = I(:,2).*I(:,3).*cos(2*pi*I(:,1));

Ycoord = I(:,2).*I(:,3).*sin(2*pi*I(:,1));

I(:,1) = Xcoord;

I(:,2) = Ycoord;

clear Xcoord Ycoord

% Project 3D data into 2D projected view from current camera view point within app

J = rotateColorSpace(I);

% Apply polygons drawn on point cloud in app

polyBW = applyPolygons(J,polyBW);

% Combine both masks

BW = sliderBW & polyBW;

% Initialize output masked image based on input image.

maskedRGBImage = RGB;

% Set background pixels where BW is false to zero.

maskedRGBImage(repmat(~BW,[1 1 3])) = 0;

end

function J = rotateColorSpace(I)

% Translate the data to the mean of the current image within app

shiftVec = [0.035456 0.001129 0.241630];

I = I - shiftVec;

I = [I ones(size(I,1),1)]';

% Apply transformation matrix

tMat = [-0.488370 -0.361528 0.000000 0.683321;

    0.012842 -0.028299 0.658630 -0.491234;

    0.282769 -0.623110 -0.029912 8.146246;

    0.000000 0.000000 0.000000 1.000000];

J = (tMat*I)';

end

function polyBW = applyPolygons(J,polyBW)

% Define each manually generated ROI

hPoints(1).data = [0.191929 0.018162;

    0.548562 -0.007794;

    0.502281 -0.306294;

    0.257266 -0.341984];

% Iteratively apply each ROI

for ii = 1:length(hPoints)

    if size(hPoints(ii).data,1) > 2

        in = inpolygon(J(:,1),J(:,2),hPoints(ii).data(:,1),hPoints(ii).data(:,2));

        in = reshape(in,size(polyBW));

        polyBW = polyBW | in;

    end

end

end

function [ocrI, results] = evaluateOCRTraining(I, roi)

% Location of trained OCR language data

trainedLanguage = '/Users/admin/Documents/MATLAB/myLang/tessdata/myLang.traineddata';

% Run OCR using trained language. You may need to modify OCR parameters or

% pre-process your test images for optimal results. Also, consider

% specifying an ROI input to OCR in case your images have a lot of non-text

% background.

layout = 'Block';

if nargin == 2

    results = ocr(I, roi, ...

        'Language', trainedLanguage, ...

        'TextLayout', layout);

else

    results = ocr(I, ...

        'Language', trainedLanguage, ...

        'TextLayout', layout);

end

ocrI = insertOCRAnnotation(I, results);

end

%--------------------------------------------------------------------------

% Annotate I with OCR results.

%--------------------------------------------------------------------------

function J = insertOCRAnnotation(I, results)

text = results.Text;

I = im2uint8(I);

if isempty(deblank(text))

    % Text not recognized.

    text = 'Unable to recognize any text.';

    [M,N,~] = size(I);

    J = insertText(I, [N/2 M/2], text, ...

        'AnchorPoint', 'Center', 'FontSize', 24, 'Font', 'Arial Unicode');

    

else

    location = results.CharacterBoundingBoxes;

    

    % Remove new lines from results.

    newlines = text == char(10);

    text(newlines) = [];

    location(newlines, :) = [];

    

    % Remove spaces from results

    spaces = isspace(text);

    text(spaces) = [];

    location(spaces, :) = [];

    

    % Convert text array into cell array of strings.

    text = num2cell(text);

    

    % Pad the image to help annotate results close to the image border.

    I = padarray(I, [50 50], uint8(255));

    location(:,1:2) = location(:,1:2) + 50;

    

    % Insert text annotations.

    J  = insertObjectAnnotation(I, 'rectangle', location, text);

end

end

As we can see in the image, all the colors have vanished, and we are only left with what we want, 1, 2, 3, in black and white. By using the color thresholding app, I inputed the function to the code. Using the same method, I used the function for the blue line (A, C, E) from the color thresholding app, and the image gave me A, C, E in black and white.

Now, the next step is to actually make this useful for a visually impaired individual. In order for us to make this useful, this image above will actually have to speak to us. So the following code will actually say the subway train 1, 2, 3, N, R, Q.

STEM Progress Meeting

We are going to have our next STEM Progress Meeting next Monday (12/12/2016). Please prepare it thoroughly and be ready for any questions. Each team will have 10 ~ 15 minutes to present depends on the size of the team. Everyone in the team should present and participate in Q & A. The outlines of the presentation are listed below.

STEM Team Progress Report Outlines
1. Topic
   a. Research topic
   b. Team member(s)
2. Research problem (if your problem stays the same, just briefly recap in one slide. If your project is refocused, elaborate it more.)
3. Method (Methods have chosen to solve the problem.)
4. Progress
   a. Gantt chart : current status & next steps
   b. Show project achievements through demonstrations (Must have!)
5. Problems & Risks
   a. Technical issue(s)
   b. Resource issues(s): tools, materials, etc. need purchase (vendor, price, and time frame)
6. References (updated Project Resource page on ....)

Every team member should have the equal opportunity to present/demo. Group and individual performance will be evaluated by peers and teacher.

Good luck!

Technical Paper Access

You can only access this if you are on a school computer. Just click that you go to Bronx Science because they have paid for a membership, so you click Bronx Science and then you will have access to the whole database.
http://www.sciencedirect.com/

Progress Meeting 1 (10/06/2016)

We are going to have our first STEM Progress Meeting this Thursday (10/06/2015). Please prepare it thoroughly and be ready for any questions. Each team will have 10~15 minutes to present depends on the size of the group. The outlines of the presentation are listed below.

STEM Team Progress Report Outlines
1. Topic
   a. Research topic
   b. group members
2. Research problem
   a. Definition
   b. Scope
   c. Functional specification
   d. Performance specification
3. Prior Arts / Resource Inventory  
   a. Survey of literature
   b. Work from last year
4. Method
   a. Possible methods/strategies to solve the problem
   b. The method (currently experimenting) & rationals of choosing
5. Technology Analysis
   a. Mechanical
   b. Electronic
   c. Algorithms 
   d. Software
   e. Chemical
    f. Biological
6. Network Diagram & Gantt chart (Time & Human Resource)
   a. Major activities/milestones
   b. Dependence
   c. Task/resource assignment
   d. Current status
7. Material Resource (Highlight Items Need Purchase)
   a. Tools
   b. Materials
   c. Software
   d. Equipment
8. Progress
   a. Learning progress
   b. Failed experiments
   c. Project achievements
   d. Demonstrations
9. Problems
   a. Technical issues
   b. Resource issues (time, human, material)
10. Safety/Risks Management
   a. Potential show stopper
   b. Safety issues.
   c. Backup plan
11. References
   a. Books/magazines
   b. Video tutorials
   c. Websites
   d. Papers from professional journals
   e. Thesis/dissertations

Project team members should coordinate with each other to create a seamless presentation. Every group member should have the equal opportunity to present/demo. Group and individual performance will be evaluated by peers and teacher.

Good luck!

Our 1st powerpoint

hyperlink attached above

STEM Seminar

We are going to have our first STEM Seminar this coming Thursday (9/15) and next Monday (9/19). Each team should prepare to present a key topic of your research field in depth based on both your study and research experience. The contents should be informative and educational to the class. The presentation should be no longer than 12 minutes (including 2 minutes for questions). You are encouraged to use PowerPoint, videos, demonstrations and handouts to help the class understand and grasp the new materials in a short time. Presentation materials should be in professional quality: concise, accurate, logical, rich in contents, and visually pleasant.

Please plan, coordinate, and rehearse your presentation in advance, and submit your topic by the end of Wednesday (9/14). The order of presentation will be announced before the presentation. If you need any special setup for your presentation, please notify Mr. Lin as soon as possible.

Once your team has finished your presentation, your presentation and multimedia files should be posted  onto your team blog. You can upload your file to Google Drive first and then share the link on your blog.

 

 

August! For Real?

Hello Everyone!

It's hard to believe that it's August! How is your summer so far? I know that some of you have summer jobs, internships, research, overseas trips, or are simply relaxing...... You are more than welcome to share your summer with us by posting on this blog. 

I have spent my past three summers in research labs, but this summer I am simply staying home and doing past-due "homework". I got an assignment to fully upgrade our old kitchen and laundry room. Although there is a skillful contractor helping us with the renovation, the project demands much more than I expected it to.

Old Kitchen View 1

Old Kitchen View 2

Torn-down Kitchen View

In addition to having to move everything out of the work area and living without both a kitchen and laundry room for over a month, I have had to design the floor plan (over 4 versions of it) and cabinet combinations according to our preferences. I have also had to shop around dozens of stores to find quality products (cabinets, counter-tops, sinks, faucets, cabinet handles, floating shelves, garbage disposer, floor tiles, backsplashes, paints, lighting, range&oven, hood, over-the-range microwave, washer, dryer, doors, locks, storm door, door thresholds, etc.) at lower prices, pick up materials, and/or arrange for shipping in a timely manner. Lastly, I also had to supervise and support workers' demands.

New Kitchen in Progress 1

New Kitchen in Progress 2

New Laundry Room in Progress

That's the reason I didn't spend enough time working on my STEM projects in the past month. However, when I reflect on what I have done so far, I realize that I am facing the same type of challenges you will encounter in your STEM projects.  

1. It's a real-world problem and will bring real impact to people's lives. Obviously, the process and results of our little kitchen & laundry room project will change our family lives. 
2. It's new to me, and there are lots of unknowns. Vague and uncertain situations are common. I had to do lots of research and read through tons of incomplete specifications/manuals/information to resolve specific issues.
3.  It's complex, and there are many factors (design, quality, availability, cost, etc.) intertwined together. There was no easy decision that could be made on almost every issue. For example, unavailable/over-priced backsplashes tiles could cause the change of the wall paint color selection. I had to understand, analyze, and evaluate all the factors before making trade-offs.
4. It requires lots of iterations to refine the solutions. The more I learned from new information or experiences, the better I could improve my project. It's a constant learning and improving process.
5.  It involves having to work within a certain time limit and a certain budget.

This is my summer so far. How about yours? Wish you all enjoy the second half of your summer! I will soon be wrapping up my little project and joining you in your STEM research activities.

Happy Research!

Bioinformatics I: Project Resource

Bioinformatics I: Project Resource: Genetics and Molecular Biology An Introduction to Genetic Analysis . Bioinformatics Bioinformatics Methods I , Coursera . Bioinforma...

Statistics 
Bayesian, ISBA
Markovian Models, Berkeley  

Summer Research Kick Off

Hi All,

Happy July 4th! At the same time, we kick off our STEM 2016-17 Summer Research!

A few announcements:

1. By now, everybody should have received your summer assignments for the first week (07/04 - 07/08). If not, please go to the class blog, and then click your team link on the right. You should be able to see your assignments. Please check whether the links are working properly and pointing to the right documents. If anything is not right, please email me ASAP. If you have any question about the assignments, please feel free to email me.
2. By the end of the week, you (and your partner if applicable) should post your progress and results on your team blog.
3. When you read the assigned materials, or watch the video tutorials, please feel free to use your team blog to take notes, record your reflections, and write down your questions. If you have not accepted the invitation as the author of your team blog, please check your email (or spam folder). If you cannot find it, email me and I will re-invite you again.
4. Each team blog has an "Project Resource" page (the link is on the right side). It has a list of project-related resources. I have started putting some links of papers, books, courses, tutorials, etc. Whenever you find something valuable for your project, you should add them to the list.
5. For people who haven't make final decision about your topic or team, you are encouraged to discuss any issues with me. You might want to browse our class and team blogs to get more concrete ideas about what other teams are working on. If you decide to form a new team, I will furnish your assignment starting week 2.

Happy Researching!

STEM Research Project 2016-17

Thanks for your responses! At the beginning of the STEM projects, it is always extremely confusing, but exciting. Unlike most of the other subjects, we don't have any textbooks, fixed course contents or tests. Problems are what we really concern; grades are never at the center of the course. Currently, you are choosing the research areas that are interesting to you, identifying the problems that you are eager to wrestle with, finding partners who sharing the same vision, and participating in the process of defining your own projects for next year. Instead of viewing this as an ill-defined stage of the course, it is actually an important part of the STEM research process! Learning how to ask good questions or forming meaningful hypotheses is a major step in research. Your focus might change, your team might regroup, and your topic might become more specific as your understanding of the research field grows. It's a brand new learning experience! So, the following list is an evolving document. As you decide or change your topics or teams, just send me an email, and I will update the list accordingly. Hopefully this process will be settled down before the summer starts, and we can kick off our summer research!

• Team 1: Autonomous Drone Navigation System in Indoor Environment (Tai wei & Ayoush)
• Team 2: Autonomous Drone Navigation & Swarm Drones (Jacob & Alan)
• Team 3: Big Data & Machine Learning (Belinda)
• Team 4: Bioinformatics I (Parvaneh)
• Team 5: Bioinformatics II (Desirae)
• Team 6: Bioinformatics III (Alex)
• Team 7: Navigation Apps for the Visually Impaired (Avleen)

Advanced STEM Research Introductory Meeting

Here is the PowerPoint file presented in our Introductory Meeting that lays the basic framework for our STEM 2016-17 course. It includes some important info about our research topics & teams, upcoming summer research activities, and the initial planning of next year. Please send me email about your grouping at the end of today. If you have missed the meeting today, please read the file carefully. We are going to use the class blog as our main communication channel. A good course can never be designed by the instructor alone. Your co-design input will make the course great! Feel free to share your thoughts and initiate any discussions on the blog. I am looking forward to having a fun and fruitful year!  

Welcome to the Advanced STEM Research 2016-17 Course

Hi All,

Welcome to the Advanced STEM Research 2016-17 course. The moment you have signed up for this course, you have also become a member of the most vibrant STEM research group in the ElRo community. Advanced STEM Research is a yearlong project-based course. The primary purpose of this course is to provide students an opportunity for firsthand, supervised research in science, technology, engineering, and mathematics fields.

In order to facilitate the activities and assignments during the summer and in the new academic year, students who have signed up for the course are required to furnish the AdvancedSTEM Research Personal Profile 2016-2017 form. Please log into your school email account before filling up the form, and don't forget to send yourself a copy of your response at the end. The form is due midnight of Wednesday (06/15/2016). Your timely and detailed feedback will maximize your opportunity to land you in a project team of best match.

In addition, we will have our MANDATORY Advanced STEM Research Meeting Thursday (06/16/2016) at 11:45 am-12:15 pm in Room 201.

Here is the agenda:

1. Project teams/topics 
2. Summer research activities and assignments
3. Resource

Some of you might need certain computer equipment to conduct your summer research. Please use the attached Computer Loan Agreement Form for borrowing computers from school, and bring back the signed form and the $200 check before the last day of school.

Since the meeting time will be very short (right between Physics Regents reviews ), we won't have too much time for detailed discussion. If you have any questions, issues, or suggestions, please email them to me ASAP. Thanks and see you Thursday!

Mr. Lin