Games@Large
The Games@Large is an ambitious EU project that aims to “research, develop and implement a new platform aimed at providing users with a richer variety of entertainment experience in familiar environments, such as their entire house, hotel room, elderly house and Internet Café. This will support ubiquitous game-play throughout such environments, while taking advantage of existing hardware and providing multiple members of the family and community the ability to play simultaneously and to share experiences.”
Interestingly (from my perspective) this is very similar to work that we are doing here at Victoria University (although with a smaller budget). We’re building two classes of tool: a platform called RemoteMe that allows games hosted on a central server to be played on a variety of Java-enabled mobile devices and a platform (nameless so far) that allows users to play old i386-based games via a standard web browser either on a desktop platform or a mobile platform.
Conflicker
Today the Conflicker worm variant C is due to start checking 500 out of 50,000 pseudo-randomly generated domain names once per day (A & B checked 250 per day) to see if any of them are happy to give it commands and to look for updates to its code. To make things even more versatile Conflicker has the ability to update other infected hosts on the network.
There has been a lot of hype about the threat posed by Conflickr and it seems to me that a lot of it is based around the fact that someone has actually taken a smarter than usual approach to building the worm and building a call-back approach that is more robust than usual (rather than trying a fixed set of domain names it has a means of generating new ones according to a pattern known to the attacker). In addition the update mechanism is quite secure and has the aim of preventing other hackers (or white hats) trying to subvert or take over the worms that are already present on machines.
Apparently, other virus makers are taking advantage of the worry about Conflicker and putting up cleaners that are really trojans. To combat this the folks at DShield have created a page where they have drawn together trusted sources in one place for information about Conflicker and tools for removing it. Also they link to the Honeypot Project Know Your Enemy article (these guys have also worked out a way to detect Conflicker remotely without needing access to the system).
From the articles it seems that aside from taking normal precautions there is nothing dramatic that will happen as the .C variant kicks off (it is at the moment in the minority but it will be interesting to see how it grows). The update mechanism is relatively lightweight so DDoS is unlikely and there is no indications that the owners of the botnet intend to do anything out of the ordinary today (there are maybe 1-2 million or more hosts in the botnet). It is likely that is business as usual for the owners (probably spam generation, porn serving etc. etc.).
GhostNet
While checking the Dalai Lama’s computer, Canadian researchers discovered a huge electronic spying operation that infiltrated hundreds of government and private offices around the world.
Their sleuthing opened a window into a broader operation that, in less than two years, has infiltrated at least 1,295 computers in 103 countries, including many belonging to embassies, foreign ministries and other government offices, as well as the Dalai Lama’s Tibetan exile centers in India, Brussels, London and New York.
The researchers, who have a record of detecting computer espionage, said they believed that in addition to the spying on the Dalai Lama, the system, which they called GhostNet, was focused on the governments of South Asian and Southeast Asian countries.
Reported by John Markoff, New York Times, March 28, 2009.
Hedy Lamarr
To celebrate Ada Lovelace day I pledged to blog about a woman in technology who I admire.
I first heard about Hedy Lamarr at a workshop meeting for a computer science research project of which I was a junior member. One of the organisers offered a prize if we could name the scientist who looked like this. I don’t think anyone actually guessed who it was although she did seem somehow familiar.
It was Hedy Lamarr who developed the idea of communication using a frequency-hopped spread spectrum while in Hollywood making feature films. She and a friend George Antheil submitted the idea of a Secret Communication System in June 1941 to the Patents office and were granted a patent in 1942. As the Wikipedia puts it “This early version of frequency hopping used a piano roll to change between 88 frequencies and was intended to make radio-guided torpedoes harder for enemies to detect or jam.”
I have to say I admire Hedy for a number of reasons. She was smart, she was courageous (she was Austrian and of Jewish parentage, she fled to Paris pretending to be her own maid) and she looked pretty cool.
R
Great article in the New York Times about R. I actually didn’t realise it was from NZ!
Credibility
Reading the chapter on credibility from Witten and Frank’s book:
“So the question is, is the error rate on old data likely to be a good indicator of the error rate on new data? And the answer is a resounding no—not if the old data was used during the learning process to train the classifier … Why? Since the classifier has been learned from the very same training data, any estimate of performance based on that data will be optimistic, and may be hopelessly optimistic.” (pg. 121)
“People … often talk about three datasets: the training data, the validation data, and the test data. The training data is used by one or more learning schemes to come up with classifiers. The vliadation data is used to optimize parameters of those classifiers, or to select a particular one. Then the test data is used to calculate the error rate of the final, optimized scheme. Each of the three sets must be chosen independently” (pg. 122)
“There’s a dilemma here: to get a good classifier, we want to use as much of the data as possible for training; to get a good error estimate, we want to use as uch of it as possible for testing.”
“The holdout method reserves a certain amount for testing and uses the remainder for training (and sets part of that aside for validation, if required). In practical terms, it is common to hold one-third of the data out for testing and use the remaining two-thirds for training … you should ensure that the random smaplng is done is such a way as to guarantee that each class if properly represented in both training and test sts. This procedure is called stratification and we might speak of stratified holdout … A more gneral way … is to repeat the whole process, training and testing, several times with different random samples. In each iteration a certain proportion – say two-thirds – of the data is randomly selsected for training, possibly with strafitication, and the remainder used for testing. The error rates on the different iterations are averaged to yield an overall error rate.”
They continue in their dicussion of measuring the rror rate of a learning scheme on a particular dataset and recommend ten times tenfold cross-validation. Here the data is divided randomly into pten parts, in each of which the class is represented in approximately the same proportions as in the full dataset. Each part is held out in turn and the learning scheme training on the remaining nine-thenths; then its eror rate is calculated on the holdout set. Thus the learning procedure is executed a total of ten times … the ten error estimates are averaged to yield an overall error estimate”. Ten is an emprircally arrived at number. Stratification will improve the results slightly. The error estimate will vary each time you run the tenfold cross-validation so it is best to run it ten times and average the results.
Easy conversion of Visio diagrams into eps
I’ve been using Visio to create diagrams. It supports a particular format that I like called Herring Bone diagrams. The problem is converting Visio diagrams into eps for inclusion in latex. Visio used to have eps export but it got broken a while back and has been removed subsequently. I’ve read some reasonably complex workarounds but found this one that seems to work for me:
- Export as a Windows Meta File (WMF).
- Use ImageMagick to convert from WMF to EPS.
- Include the file or convert to PDF if using pdflatex.
That’s it. Seems to work, which is a big suprise!
Most experiments are local but have general aspirations
Cronback noted that each experiment consists of units that receive the experriences being contrasted, of the treatments themselves, of observations made on the units, and of the sessings in which the study is conducted. Taking the first letter from each of these four words, he defined the acronym utos to refere to the “instances on which data are collected” — to the actual people, treatments, measures that were sampled in the experiment. He then defined two problems of generalization: (1) generalizing to the “domain about which [the] question is asked”, which he called UTOS; and (2) generalizing to “units, treatments, variables, and settings not directly observed”, which he called *UTOS. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Shadish, Cook and Campbell. 2002.
There are two types of generalization that may be an issue in experimental design: (1) Construct validity — are the labels applied to the elements of the experiment (UTOS) really represent the concepts that appear in the underlying theory that we wish to test. (2) External validity — either from one unit to another (for example, generalizing from one city to another) or broader (for example, where we randomly sample we may be able to generalize from our sample to the population being sampled).
Preliminary Hazard Analysis
… is used in the early life cycle states to identify critical system functions and broad system hazards. The identified hazards are often assessed and prioritized, and safety design criteria and requirements may be identified … Because PHA starts are the concept formation stage of the a project, little detail is available, and assessments of hazard and risk levels are necessarily qualitative and limited. (Nancy Levenson, Safeware)
This applies to our project. It has been unclear how to rank the different liklihoods of a risk to the experiment.
Reinvention
DP: As you moved through your career, what was the smartest decision you ever made?
GC: The best decision – and it’s a sequence of decisions that I’ve made – is that I reinvent my technical area every five or six years.
Typically, that’s the lifespan of technical areas. You start with some new ideas and it’s really interesting and important to develop those ideas. But, over five or six years, the areas mature. There’s still stuff to do but it’s more second-order kinds of problems, different kinds of problems from the ones I like doing, which are more fundamental, more foundational.So, the smartest thing is that I try not to get stuck in one particular area; that I try to reinvent myself consciously. That’s what’s kept me going the last few years.
George Cybenko, Dorothy and Walter Gramm Professor of Engineering at Dartmough College, inverviewed in 2008.
Opus iti -- my little work 
