... and logistic regression

A follow-up on the collaborative filter, it occurs to me that it should be possible to add a logistic regression layer in Keras. The following is a regularized multinomial logistic regression model: from keras.models import Sequential from keras.layers.core import Dense, Activation from keras.regularizers import l2 model = Sequential() model.add(Dense(numClasses, input_shape=(numFeatures, ), \ init='zero', W_regularizer=l2(0.01))) model.add(Activation('softmax')) model.compile(loss='binary_crossentropy', optimizer='sgd') As with the collaborative filter, you can easily modify the code to use multiple regularizers and different learning algorithms, say Adamax or adam instead of SGD. ...

March 8, 2016 · 1 min · admin

Collaborative filtering in Keras

Ten years ago, Netflix started the Netflix challenge. A contest to see if the community could come up with a movie recommendation approach that beat their own by 10%. One of the primary modeling techniques that came out of the contest was a set of sparse matrix factoring models whose earliest description can be found at Simon Funk’s website. The basic idea is that the actual ratings of movies for each user can be represented by a matrix, say of users on the rows and movies along the columns. We don’t have the full rating matrix, instead, we have a very sparse set of entries. But if we could factor the rating matrix into two separate matrices, say one that was Users by Latent Factors, and one that was Latent Factors by Movies, then we could find the user’s rating for any movie by taking the dot product of the User row and the Movie column. ...

March 7, 2016 · 3 min · admin

The Sounds of Science

Last Friday, hsarik pointed out an interesting web site: Echo Nest. They provide a web service that allows you to analyze and remix music. The API also can provide information (meta-data) about music, artists, songs etc. and has Python bindings. If you’ve seen the “More Cowbell” website where you can upload an mp3 and have more cowbell (and more Christopher Walken) added to it, well that site uses Echo Nest and if you download the python bindings for their API, you can see the script that adds the sounds. Personally, I’m fond of “Ob-la-di, Ob-la-da” with 80% cowbell and 20% Christopher Walken. ...

June 17, 2010 · 3 min · admin

Model update (updated!, updated again)

Earlier, I posted my current model for predicting the NCAA tournament. Since the whole thing is probabilistic, I figured that I would test it out against the current NCAA standings. I considered four models: The one that I described A random selection of which team would win (50/50 chance) Always picking the top seeded team A model suggested by a colleague at work For each model, I ran 10,000 tests and compared them to the current NCAA tournament results, counting the scores for each test. Results are: ...

March 23, 2010 · 2 min · admin

They laughed at my theories!

They laughed at my theories. They threw tomatoes when I presented my paper at the academy! Tomatoes I tell you! My minions cower in terror, shrinking in fright from the very ideas contained herein! But I will show them! I will PROVE IT TO THEM ONCE AND FOR ALL. The FOOLS, I WILL DESTROY THEM!! MWAHAHAHAAAA!(ask me how) Oh, sorry. Where was I? Apparently, there’s this basketball thing going on. Some sort of NCAA tournament that will prove who has the best basketball team. But what if it doesn’t? What if it’s all just arbitrary? Could it be that the chances of any team winning a game are not deterministic, but rather stochastic? I’ll admit that I don’t know that much about basketball. I mean, I played the sport in junior high. I do know the rules. And I even think that it’s a pretty game. But I don’t follow the ins and outs of a particular season. ...

March 22, 2010 · 4 min · admin

"Hacking" predator drones

This just makes me sad. Two articles, one in the WSJ, the other on CNN, describing how insurgents in Iraq are hacking predator drones and receiving the video feeds that the drones are sending back to U.S. ground stations. First things first, let’s fix the headlines. Both are running something like “Iraqi insurgents hacked Predator drone feeds.” That should more clearly read: “Iraqi insurgents watching the videos that the Predator drone sends out unencrypted.” Or maybe “Iraqi insurgents watch Predator drone feeds on TV.” ...

December 18, 2009 · 3 min · admin

Trash heap of programming history

When we put in the massive wall-eating bookcase, I thought that we had enough book space for the next 10,000 years! Or at least the next 10. Unfortunately, things were filling up a bit too fast, so I grabbed some of the books that I know I’ll never use again and will send them to recycling. Most are completely out of date (from bottom to top, these are from the mid-80s to the late 90s). Linux Application Development is probably still relevant, but a little too basic. Linux Device Drivers goes all the way through the 2.0 kernel series (and may be relevant for the experimental 2.1 series)! Switched LANs (snicker) and In Search of Clusters were given to me by vendors. And if I ever have to program native X-Windows again, I’ll kill myself. I kept the books on C/C++, Python, Perl and PHP - though I’ll probably never buy another programming book. Maybe Celeste is right, I should just get a Safari account… Google works pretty well too. If there’s anything you want in here, let me know… 10258

November 29, 2009 · 1 min · admin

Great moments in . . .

Minor notes, none worth their own post. Traffic management: I get a call from K around 5:30. She’s stuck behind an accident and the cops on the scene, a) don’t tell people to take a detour until they’ve been there for a half hour; and b) once the ambulance has left the scene, don’t direct traffic around the one remaining open lane. So, after waiting a half hour, K has to take a 20+ minute detour home. Memory: Once she gets in, K and I are fixing leftovers for dinner. C: “Hey, where are the mashed potatoes?” K: “Where did you put them?” “In the fridge, but I can’t find them.” “Maybe they’re in the freezer.” “Nope, not there either.” Ten minutes of looking for the potatoes. Did we throw them out on Sunday? Nope, not in the trash. Did C put them in the pantry? Nope. Can’t find ’em, can’t find ’em. Finally, K says, “wait, we fixed rice on Sunday.” There weren’t any potatoes. I would attribute it to getting old, but I’ve always been this way. FUD (fear, uncertainty and doubt): we’re testing some things at the office – will our authentication system (active directory) honor password failure lockouts when using LDAP authentication? I ask our windows consultant to either a) answer the question, or b) enable an account lockout policy so we can test. He responds back that he can do that, but with the warning that “many Linux services aren’t well-designed for this, and repeatedly try a cached or user-provided password, so that users or service accounts may be mysteriously locked out after one attempt or at some future time when passwords change.” Which is complete and utter B.S. Signs that it’s BS? He references Linux services as opposed to open source, i.e. attempted linux dig. And I used to “own” identity management services, including authentication at a large university and if this was the case, things would have blown up within 10 minutes. I thanked him for the advice and noted that I’ve never seen this, but that it’s why we test. OS Performance: we’re looking into some new ideas at the office. Things that could be useful as a preprocessor for a host based intrusion detection system. As part of my testing, I told my laptop to audit all syscalls made to the kernel, by all processes on the system. CPU load spiked, system performance went through the floor, the windowing system became almost completely non-responsive. In the two minutes it took to get access to a terminal, I logged 150 MB of audit logs. On the plus side, all of the information we need can be collected. Now I just need to figure out how to keep a usable system. Self aggrandizement: talking to my technical manager, we need to write up two journal papers based on our recent work. Cool! I hope everyone had a good Veteran’s Day and remembered to thank the veterans in their lives.

November 12, 2009 · 3 min · admin

Facebook security vulnerabilities

and this is why I like cross-posting to facebook from my blog. It’s a healthy reminder that nothing on fb is actually private. If it’s online - it’ll be exposed eventually, whether through a new exploit, or just because you “friend” someone in the future that you had written about in the past. h/t hsarik

November 5, 2009 · 1 min · admin

Toy!

For a guy who uses computers as much as I do, you would think that I would have better machines at home. But really, no. Our last desktop was eight years old before we replaced it. My laptop was an old Thinkpad X31 - maybe six or seven years old. So, for a combination birthday/Christmas present, I got a new (sort of) laptop. I picked up a refurbished thinkpad X200s from the Lenovo outlet. So far it’s a nice machine: good processor, lots of high speed memory, and a big LED backlit screen. Lenovo sent me notice that they were shipping the laptop today and the next thing I knew, it arrived at the office. The advantages of working 5 miles from the outlet. The only problem is that they didn’t ship all of the parts. I’ve contacted them and they’ll send the dock/dvd. ...

November 4, 2009 · 1 min · admin