Posts with «computer science» label

Facebook is using first-person videos to train future AIs

One of the obvious goals of almost every computer vision project is to enable a machine to see, and perceive, the world as a human does. Today, Facebook has started talkingabout Ego4D, its own effort in this space, for which it has created a vast new data set to train future models. In a statement, the company said that it had recruited 13 universities across nine countries, who had collected 2,200 hours of footage from 700 participants. This footage was taken from the perspective of the user, which can be used to train these future AI models. Kristen Grauman, Facebook’s lead research scientist, says that this is the largest collection of data explicitly created for this focus.

The footage was centered on a number of common experiences in human life, including social interaction, hand and object manipulation and predicting what’s going to happen. It’s, as far as the social network is concerned, a big step toward better computing experiences which, until now, have always focused on sourcing data from the bystander’s perspective. Facebook has said that the data sets will be released in November, “for researchers who sign Ego4D’s data use agreement.” And, next year, researchers from beyond this community will be challenged to better train machines to understand what exactly humans are doing in their lives.

Naturally, there is the angle that Facebook, which now has a camera glasses partnership with Ray Ban, is looking to improve its own capabilities in future. You probably already know about the perils of what this potential surveillance could entail, and why anyone might feel a little leery about the announcement.

Embedded programming gap

It seems that there is a shortage of programmers who can do embedded systems (which is what computer engineering is mostly about these days).

Critics lay much of the blame for the embedded programming gap at the doorstep of university computer science departments that have tended to migrate curricula toward trendy programming languages like Java at the expense of unglamorous tasks such as how to design and analyze algorithms and data structures.
Struggle continues to plug embedded programming gap | EE Times (by George Leopold)

I’m not so sure that Java is at fault here. It seems to me to be perfectly fine second programming language (after a simpler one like Python that does not require all data structures to be declared before any code is written).  The problem is more that the instruction focuses entirely on designing huge complex data structures and using large libraries of complex software components, rather than on fundamentals:

The problems start early in the curriculum. Too often an introductory Computer Science course will fall into one of two extremes: either focusing on the syntactic details of the programming language that is being used—“where the semicolons go”—or else treating programming as a matter of choosing components and plugging them into a GUI.

The basics—how to design and analyze algorithms and data structures—are covered summarily if at all. Software methodologies such as Object Orientation that are best approached in the context of the development life cycle for large applications, are introduced through trivial examples before their benefits can be appreciated: the proverbial cannon that is used to shoot a fly.

The education of embedded systems software engineers: failures and fixes | EE Times (by Robert Dewar)

I’m not so sure that I believe in Robert Dewar’s proposed solution, though, as he suggests having students do more high-level design (software architecture, rather than nuts-and-bolts programming), which is in direct opposition to his claim that students should be getting more training in low-level languages like C.

Robert Dewar also makes an argument for group work at the university level—something that is common in computer engineering programs, but apparently rare in computer science programs.  At UCSC, I know that all computer engineers, electrical engineers, and game design majors are expected to do group senior projects, and some of their other classes (such as mechatronics) are also group projects.

I think that the lack of group projects in many CS courses is not so much tied to Dewar’s idea “a perhaps regrettable staple of the educational process is the need to assess a student’s progress through a grade, and a team project makes this difficult” as it is to the problem of scale—a group project is only reasonable when the project is too big to be done more efficiently by a single person.  Creating and managing such big projects in lower-level classes would be a major undertaking, particularly in the short time frame of a quarter or semester, when a lot of things other than group dynamics need to be learned. Pasting a group structure onto tiny project would make things worse, not better, in terms of training students to be effective in groups (see Group work).

Some entrepreneurs have addressed the problem by starting up “initiatives like Barr’s week-long, hands-on Embedded Software Boot Camp.”  The idea is to take programmers who already have degrees and supposedly know C and train them specifically in the skills needed to do real-time programming. The cost is not small ($3000 for 4.5 days, if you register early).

Some computer scientists have been pointing out problems in the standard CS curriculum for a while:

I started saying this over a decade ago. I even did embedded stuff in my 3rd year data architecture course—my department was uninterested, and the students had a real hard time wrapping their heads around the thought that there are places where resources are limited.

The department fought me when I said that students needed to learn more than one language (Java). The department disagreed when I said that students should learn how to program for environments where bloated OO methods might not work (… But, the ARE no places where efficiency matters!!! It’s all about “Software Engineering”!).

The students had NO idea what it meant to program for a machine that had no disk, only memory.

Part of the reason CS departments are seen as being so out of touch is BECAUSE THEY ARE!!!

University should not be about job training, BUT it is also NOT about teaching only those things the faculty find interesting.

Struggle continues to plug embedded programming gap | The Becker Blog.

I know that there have been struggles between the computer science and computer engineering departments at UCSC about what programming language to teach first, with the computer scientists arguing for Java and the computer engineers arguing for C and assembly language.  Eventually they reached a compromise (which has been stable for about a decade), with the computer science students taught Java first and the computer engineering students taught C first, then both making transitions to the other language.

I think that both approaches work, but the strengths of the resulting programmers are somewhat different.  For device drivers and small embedded systems, I’d bet on the computer engineers, who have a better grasp of low-level details, interrupts, and hardware/software interactions.  For more complicated projects, I’d want one of the computer scientists doing the high-level programming, teamed with computer engineers to do the detail work.

I actually don’t like either C or Java as a first programming language.  I think that students would be better off starting with Scratch, which gets them quickly into multi-threaded code, real time operation, and race conditions, without burdening them with a lot of syntax.  Then they can switch to Python to learn about code syntax, types, and objects, without the burden (or support) of compile-time type checking.  After that they can switch to Java to learn about declaring all their data structures and having very rigid type rules (useful for bigger projects to make interfaces between different design groups more explicit).  In parallel with learning Python and Java, they should learn C and C++ in the context of the Arduino microprocessor (or similar small microprocessor) to control real-time physical systems.

The computer engineers could even skip Java, and learn their rigid type systems strictly in C++ (which they are more likely to use later), though Java is cleaner and makes the learning somewhat easier.

Computer scientists should not stop with this handful of languages, though, as they are essentially all Algol derivatives.  The CS students should also learn a couple of languages that come from different lineages (LISP or Haskell, for example).


Filed under: Uncategorized Tagged: Arduino, C++, computer science, education, higher education, Java, programming, Python, Scratch

Scratch plus Arduino

Small S4A (Scratch for Arduino) program that detects when a button is pressed. I used a pull-up resistor, so pressing the button pulled the signal low. Since that is a very standard circuit, I object somewhat to the "Sensor pressed" terminology. It should say either "Sensor Digital1" (since the value is Boolean) or "Sensor Digital1 high"—the "pressed" word is misleading.

There is a project now that combines two of my favorite tools for teaching about computers: Scratch for beginning programming and Arduino for beginning embedded systems and microcontrollers: Citilab – Projecte Scratch.  The project appears to be from Spain, as the main page of the site is in Catalan, though the Arduino page is in English.

The idea is a simple one: the Arduino is loaded with a standard program that interchanges information with the Scratch program via USB every 75 msec.  The I/O pins of the Arduino have a standard configuration:

The configuration offers 6 analog inputs (analog pins), 2 digital inputs (digital pins 2 and 3), 3 analog outputs (digital pins 5, 6 and 9), 3 digital outputs (pins 10, 11 and 13) and 4 special outputs to connect Parallax continuous rotation servomotors (digital pins 4, 7, 8 and 12).

They had to modify Scratch a little to add blocks for the I/O (similar to the official Scratch blocks for the “Scratchboard” which is not nearly as versatile as the Arduino).  It would be nice if the S4A modifications made their way into the official version of Scratch, as the Arduino and Scratch communities are natural allies in making computer science and computer engineering more accessible to non-specialists.

There are some mistakes they made.  For example, they put all the Arduino blocks with the “motion” blocks, though it would have made more sense to have a new page of blocks (or to have spread them between the motion and sensing pages).

They also refer to digital inputs pin 2 and 3 as Digital1 and Digital2, and to Analog pins 0 through 5 and Analog1 through Anlaog6.  I think that they need to make their Scratch programs consistent with the labeling on the Arduino board!  The digital outputs are correctly labeled 10, 11, and 13, but they have separate blocks for On and Off, which is not as nice as a single block that can set the digital output to a Boolean (though Scratch would have to have constantsandfor that to work well.

I’ll be sharing S4A with the robotics club, as it looks like a fun thing to play with, even if there are some minor design flaws.


Filed under: Scratch, Software Tagged: Arduino, computer science, education, programming, robotics, Scratch