The Beautiful Mind

How can I start preparing for GSoC 2014? As of now I am a novice coder.

Thu, 30 May 2013 13:32:25 -0400

Answer by Rajath Shashidhara:

According to me,
being a GSoC’er is about grasping code quickly and learning independently.
Follow these steps:

1. Install Linux. Master using the shell. Learn all the things on you own. Google stuff.
2. Choose a programming language. Work on it.Understand the general structure of it. In the sense that, if you are given a new set of API tomorrow and asked to code using it, you should be able to read the documentation of the api and use it in your code.
3. Once you have mastered the art of independent code learning, choose an opensource organization. Subscribe to their mailing list. Introduce yourself as a newbie to opensource development. Don’t be shy. My experience with opensource community is that they are very supportive. They are very helpful. You will experience a few frustrating days. Don’t give up. Setup the development environment. Then browse the buglist for bugs that catch your eye, thats looks like that can be reproduced by you. Browse the code, look for something called devguide which gives the purpose of each variable or method. Then try to relate the code to the bug. Get help from the community at each step. They will guide you. At times this can get very frustrating, be focused. Get used to bug fixing. If at any point of time, if you feel that you are not enjoying this challenge, then probably you should rethink about it and make a wise decision. This way you can get used to bug fixing. Version control is another important thing. You should learn about them and use them. Once you have done this you have reached a higher level. Now you are an opensource contributer. Congrats! Bond with the developer community.

Choosing an organization must be done smartly. If you have done the above things for a considerable amount of time, you have established as an opensource developer. Then, getting into GSoC will not be difficult at all.Your community will be happy to accept you . :)

Read my blog post: My first step into OpenSource Development. My Explorations for the detailed experience.

View Answer on Quora

How is asynchronous IO implemented in programming languages?

Sat, 06 Apr 2013 18:00:36 -0400

Answer by Prakash Gamit:

There are five types of I/O models

Diagrams are for network I/O, but they would be similar for disk I/O or any other I/O operation.

1. Blocking I/O model
Block if request cannot be completed immediately.

2. Nonblocking I/O model
Do not block if request cannot be completed immediately, return error(
EWOULDBLOCK
) instead.
Setting
O_NONBLOCK
flag using
fcntl
This is often a waste of CPU time as it uses polling.

3. I/O Multiplexing model
Call
select / poll
and block in one of these system calls, instead of blocking in the actual I/O system call.

4. Signal-driven I/O model
Tell kernel to notify application with
SIGIO
signal when the descriptor is ready.

5. Asynchronous I/O model
telling the kernel to start the operation and to notify application when the entire operation (including the copy of data from the kernel to application buffer) is complete.
Following function are available in C for asynchronous I/O in Linux 3.5 system(might be available in earlier versions also)

aio_read(3)     Enqueue a read request. This is the asynchronous analog of read(2).

aio_write(3)    Enqueue a write request. This is the asynchronous analog of write(2).

aio_fsync(3)    Enqueue a sync request for the I/O operations on a file descriptor. This is the asynchronous analog of fsync(2) and fdatasync(2).

aio_error(3)    Obtain the error status of an enqueued I/O request.

aio_return(3) Obtain the return status of a completed I/O request.

aio_suspend(3) Suspend the caller until one or more of a specified set of I/O requests completes.

aio_cancel(3)   Attempt to cancel outstanding I/O requests on a specified file descriptor.

lio_listio(3)   Enqueue multiple I/O requests using a single function call.

The aiocb (“asynchronous I/O control block”) structure defines parameters that control an I/O operation. An argument of this type is employed with all of the functions listed above. This structure has the following form:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
#include <aiocb.h>
           struct aiocb {
               /* The order of these fields is implementation-dependent */
               int             aio_fildes;     /* File descriptor */
               off_t           aio_offset;     /* File offset */
               volatile void  *aio_buf;        /* Location of buffer */
               size_t          aio_nbytes;     /* Length of transfer */
               int             aio_reqprio;    /* Request priority */
               struct sigevent aio_sigevent;   /* Notification method */
               int             aio_lio_opcode; /* Operation to be performed;
                                                  lio_listio() only */
               /* Various implementation-internal fields not shown */
           };
           /* Operation codes for 'aio_lio_opcode': */
           enum { LIO_READ, LIO_WRITE, LIO_NOP };
When any of the above function is called it creates a new thread and returns to user process immediately. This new thread keeps waiting for I/O operation to complete while the user process continues its execution. When I/O operation completes it notifies user process by delivering signal specified in aiocb struct.

The current Linux POSIX AIO implementation is provided in userspace by glibc. This has a number of limitations, most notably that maintaining multiple threads to perform I/O operations is expensive and scales poorly.

Sources:
1. Unix Network Programming, Volume 1, 3rd edition, W. Richard Stevens(Chapter 6)
2. aio(7) (aio man page)

View Answer on Quora

How do you explain basic terminology of Git in the easiest way?

Fri, 22 Mar 2013 06:14:08 -0400

Answer by Daniel Kinzler:

Lots of pretty diagrams here! Let me add mine to the mix :)

View Answer on Quora

Where can I learn more about cool (novel?) data structures and algorithms?

Mon, 18 Mar 2013 12:48:32 -0400

Answer by Aditya Jain:

Cosmin Negruseri has already given a good list that covers most of the books I will be mentioning .While answering this question , I am assuming that you are familiar with most of the material in Introduction to Algorithms by Cormen et al.

From here on, depending on your specific needs and interests , you can pick up any of the following books

1. Advanced Data Structures
Foundations of Multidimensional and Metric Data Structures (The Morgan Kaufmann Series in Computer Graphics): Hanan Samet: 9780123694461: Amazon.com: Books
This book probably has every data structure you would ever need.The book is huge so you might just want to read the relevant sections.

2.Randomized Algorithms
Randomized Algorithms: Rajeev Motwani, Prabhakar Raghavan: 9780521474658: Amazon.com: Books
Probability and Computing: Randomized Algorithms and Probabilistic Analysis: Michael Mitzenmacher, Eli Upfal: 9780521835404: Amazon.com: Books

3.Approximation Algorithms
Approximation Algorithms: Vijay V. Vazirani: 9783642084690: Amazon.com: Books

4.Computational Geometry
Amazon.com: Computational Geometry: Algorithms and Applications (9783642096815): Mark de Berg, Otfried Cheong, Marc van Kreveld, Mark Overmars: Books

5.String Algorithms
Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology: Dan Gusfield: 9780521585194: Amazon.com: Books

6.Algorithmic Game Theory
Algorithmic Game Theory: Noam Nisan, Tim Roughgarden, Eva Tardos, Vijay V. Vazirani: 9780521872829: Amazon.com: Books

7. Distributed Algorithms
Distributed Algorithms (The Morgan Kaufmann Series in Data Management Systems): Nancy A. Lynch: 9781558603486: Amazon.com: Books

View Answer on Quora

Why isn't supervised machine learning more automated?

Thu, 14 Mar 2013 08:03:37 -0400

Answer by Alex Clemmer:

In order to address the stated question (“Why isn’t supervised machine learning more automated?”), we have to begin to understand how complicated supervised learning really is. (We’ll get to the other questions in the “details” section in a minute.)

Here’s how complicated it is.

Say you work at a lab somewhere. Your boss, Professor George O’Jungle, hands you a box marked “data”. He mumbles, “use machine learning to classify this data”, and then walks away. Here is a nifty picture:

fig 1: George pictured here in skirt due to it being International Subvert Patriarchy Day

You’re pretty busy, and groan out loud:

Q: Is there some way to automate this learning task?
A: No, at least not at this point in time. You don’t even know what’s in the box. It could be hamsters or something equally ridiculous. Before we do machine learning, we must know what our observable phenomena is.

Ok, so you decide to find out. You open the box. Inside are papers with numbers on them.
Great, the empirical phenomena are something sane. If it was, like, bird migration patterns, we’d have to change real observations about that into useful learnable data. But here we have just a bunch of numbers. Computers are good at looking at doing stuff with numbers, so it looks like we’re getting something for free.

So now you ask:

Q: Now can I automate this learning task?
A: Still no. Your data are just numbers on a paper. In order to do machine learning, there must be a meaningful interpretation of our data. Interpreting the data is often called “cleaning” the data. Sometimes this step is trivial. For example, if your data are JPEG images, then you should probably just “interpret” each binary as a JPEG image. In other cases, it is less straightforward. For example, if your data are emails, then you’ll want to remove the headers, HTML tags, images, and other vestigial data that could needlessly harm your algorithm. In other words, you are “interpreting” the data as a set of emails, where “email” is really just “text in the body” or something. Not doing this correctly can literally ruin your ability to do machine learning, so don’t ignore it!

Anyway, you now set off to find out how all these pages of numbers should be interpreted. You ask Professor George. He explains that they are images. He shows you how to use his expensive image-interpreting machine. You feed all the papers into this image-interpreting machine:
[Cow image sources: File:Cow female black white.jpg, File:Cow-IMG 2050.JPG]

Now you have a nice stack of images. So once again, you come back to your original question:

Q: Ok, now can I automate this learning task?
A: Sorry, still no. You may have a useful interpretation of your data, but you have not specified your hypothesis set. A hypothesis basically maps things to some set of classes (i.e., it “classifies” things), and a hypothesis set is the set of possible hypotheses. Examples of hypothesis sets are the SVM and the perceptron. Examples of a hypothesis are weight vectors that “classify” your data. Note that hypothesis sets make assumptions about the data (for example, independence assumptions), so choosing the right model is often a balance between hypotheses that are (1) tractable to learn, and (2) expressive. In order to do machine learning, you need to know which hypothesis class you’re using, what your learning algorithm is (e.g., gradient descent), and what classes you’re mapping to.

So once more you consult Professor O’Jungle. He explains that your task is to separate pictures into piles of those that contain cows and those that do not. You can use whatever hypothesis set and learning algorithm you like. “Great!” you think. “I choose the SVM, with whatever learning algorithm is fastest, and the rest will be easy.” You sit back and grin.

Q: … Because now I can completely automate this learning task, right?
A: Wrong. You may have cleaned data, and you may have have a task, but you did not specify which features of the images are relevant to learning this task. The reason is that your hypothesis set is making assumptions about your data. In particular, most hypothesis sets designed for classification assume that your data is a -dimensional real vector, which can be interpreted as points in -dimensional space that need to be “separated.” Why and how is a technical question for another time. The question for now is: how do we take images and turn them into vectors?

You head back yet again to prof. O’Jungle. Your consolation seems to be that we must be most of the way to automating the learning by now. Or you grimly hope so anyway. You ask what features he wants to use. He slaps his forehead and exclaims that he is an idiot for forgetting to show you his feature-extract-o-tron. It’s not really automatic, because someone had to make it especially for this problem. And in general, it is not clear how to automatically select these features. One good attempt is convolutional deep belief nets, but they’re not quite “there” yet. And anyway, while this solution isn’t really automatic, it is automatic for you. And that’s good enough.

You slump back at your desk. This had better work. You feed all the images in and get back some nice vectors of real numbers:

Finally, you think:

Q: Ok, surely now I can automate this learning task, right?
A: Sorry, but no. Now you have to do the parameter tuning.

The good news is that this is pretty much automatic. First you randomly split your data into “training” and “testing” sets. Then pick a set of values that seems reasonable for your parameters. Then you train your model on the “training” data. Then you test the model on the “test” data. Finally, you select whichever model had the best results.

There probably isn’t a way to completely automate this step away in general, but it’s already pretty close.

So you end up with a well-tuned model a nice result. You look back at the process. None of the steps were really automatic, and in fact, it seems like it would be pretty hard to automate any of them.

You grumble to your colleagues about this:

Why do machine learning at all if it can’t be completely automatic?
And the answer is simple: because it actually saved you a lot of time already. Imagine if you had tried to program your cow detector by hand, without using ML. Sure, it’s not completely automatic, but no free lunch, right? (Actually it is provable that there is no free lunch, but that’s for a different post.)

You might complain that this is how you know machines aren’t “really intelligent”, but the bottom line is that if we didn’t have ML, we would not be able to quickly build things like search engines or cow detectors. That is, we could build them, but we’d build them much slower.

So: we use ML because it helps us to quickly deal with large volumes of arbitrary data. It isn’t magic, but it works well enough.

Misc: the “details” section of the question:
The question details ask why we can’t completely automate (1) parameter tuning and (2) feature selection. In the case of (1), the short answer is that parameter tuning is already the most automatic thing about supervised ML — you get a working classifier, then optimize it by running it over some range of parameters, and then select the “best” model. Easy. (2) is much harder, and the short answer is that we’re still working on automatic feature selection. Convolutional deep belief nets have great promise, but it will be many years before this is viable, if it ever turns out to be.

If you thought this was useful and you enjoy ML, you might also enjoy my Quora blog, which is primarily concerned with ML-ish problems.

View Answer on Quora

How do Go, Scala and Julia compare with each other?

Tue, 05 Mar 2013 22:11:51 -0500

Answer by Eric Talevich:

Go is a systems programming language. Google created it to safely solve three specific problems that C++ was biting them with: concurrency, memory management and compilation time on large systems. It’s concise and readable, like Scala and Julia, but a bit fussy and low-level, unlike Julia. You would use it to write safe, well-performing services that your main application might call to. It would certainly be possible to write scientific software in Go, much as you would with C or C++, but it doesn’t have the same flexibility and interactivity as, say, Python or Julia, or even Scala. If you’re trying to use Go for data analysis, the fussiness and lack of interactivity will disappoint you.

Julia is a scientific applications programming language, designed to be a good replacement for Matlab and Python+SciPy. It’s very new, so I won’t attempt to predict which specific features it will or won’t pick up, but the underlying motivations should drive it to be a very good programming language for scientific software — not syntactically optimized for statistical operations on data arrays like R, but good to write a computationally intensive program that uses multiple CPUs and generates pretty graphics.

Scala is a general-purpose language that sits between the other two, capable of dealing with either end of the spectrum. Twitter uses it for services. Scientists use it for aligning protein structures. It’s a good language, a few years older than the other two, and is built on the JVM, which has its own pros and cons. If you want your code to be something other programmers can use and/or build on, Scala is your best choice at the moment — it creates .jar files which can be used directly from plain Java, whereas Go doesn’t seem to be capable of creating reusable shared object (.so) files yet.

View Answer on Quora

Kafka writes every message to broker disk. Still, performance wise it is better than some of the in-memory message storing message queues...

Tue, 26 Feb 2013 09:57:31 -0500

Answer by Jay Kreps:

There are a few reasons:

The first is that Kafka does only sequential file I/O. To enable this kafka enforces end-to-end ordering of messages in delivery. This means the consumer has a single position in that message stream, and this position can be stored lazily. Typically messaging system keep some kind of per-message state about what has been consumed and have to update it. This introduces all kinds of random updates to mark messages consumed. By contrast Kafka keeps a single pointer into each partition of a topic, rather than a per-message state. All messages prior to the pointer are considered consumed, and all messages after it are consider unconsumed. This eliminates most of the random I/O in acknowledging messages, since by moving the pointer forward many messages at a time we can implicitly acknowledge them all. As a side benefit retaining order is good for other reasons (often the ordering has meaning). The reason most messaging systems don’t do this is because it is hard—it requires co-ordination among the consumers to “elect” consumers for each partition. We lean on zookeeper to manage this process of matching consumes to partitions of data on servers and keeping this matching up to data as the set of available consumers and brokers changes.

The second reason is because Kafka supports end-to-end batching of messages. Computers love linear scans and transfers with big arrays, they hate little bursty random messages. One prerogative of an asynchronous messaging system is the ability to introduces just a little delay to allow what would have been small bursty messages to turn into big fat ones. This speeds up network transfers, disk operations, and even in-memory iteration. We expose this as tunable parameters, so people who can stand a little extra latency can get a lot of extra throughput.

Finally Kafka leans heavily on the OS pagecache for data storage. Although the question says that kafka writes to disk immediately, that is not completely true. Actually Kafka just writes to the filesystem immediately, which is really just writing to the kernel’s memory pool which is asynchronously flushed to disk. There are a couple of reasons this is a good idea:
Kafka runs on the JVM and keeping data in the heap of a garbage collected language isn’t wise. There are a couple of reasons for this. One is the GC overhead of continually scanning your in-memory cache, the other is the object overhead (in java a hash table of small objects tends to be mostly overhead not data).
Modern operating systems reserve all free memory as “pagecache”. Basically contiguous chunks of memory that soaks up reads and writes to disk. The nice thing about this is that on a 32GB machine you get access to virtually all of that memory automatically without having to worry about the possibility of running out of memory and swapping.
Unix has optimizations to allow you to directly write data in pagecache to a socket without any additional copying (aka sendfile). Any data sent on a socket has to cross the process/kernel memory boundary any way. This means if you keep data in your process, and need to deliver that data to multiple consumers you need to recopy it into kernel space, buffering on both sides, each time. This approach gets rid of all the buffering and copying and uses and single structure.
If you are interested in this stuff there are a couple of more detailed write-ups. There is a more complete design document that discusses the trade-offs in more detail [1]. There is also a more recent write-up on the use of kafka at LinkedIn which gives some performance and operational statistics [2].

[1] http://incubator.apache.org/kafka/design.html
[2] http://sites.computer.org/debull/A12june/A12JUN-CD.pdf

View Answer on Quora

Which vs. That

Sun, 24 Feb 2013 18:32:33 -0500

Post by Gayle Laakmann McDowell:

Which vs. That

View Post on Quora

The difference between e.g. and i.e.

Sun, 24 Feb 2013 18:32:14 -0500

Post by Gayle Laakmann McDowell:

The difference between e.g. and i.e.

View Post on Quora

What are some common Machine Learning interview questions?

Sat, 23 Feb 2013 11:16:08 -0500

Answer by Charles H Martin:

When I am asked to interview people, I try to ascertain whether they know the math or not, and how to apply it in a real world context. I also look to see if they understand high performance computing and not just vanilla coding.

I was asked to do this as a consultant, acting as a subject matter expert to help interview junior people for the firm.

In our interviews, we asked a candidate to present some code they had written and to talk through it.   For an ML person, it would be some kind of ML code.

So, for example, I was involved with an interview with a Physics PhD from MIT discussing some NMF code he wrote in javascript.   The javascript was very good and he would be fine doing GUI work , Node.JS work, etc. Certainly not something I could do.

Can he do Machine Learning. Mind you, he has a PhD in a math heavy subject from one of the top 10 schools in the world. So he should know the math.

I wanted to see if he knew how to get it to converge properly. He did not.     He knew it was non-convex, but he did not know how to seed it, nor did he know about the convex variants. He tried to give me some nonsense about it being Bi-convex and whatnot. Dude, just use Kmeans++ to seed it. Thats it. Thats all you had to say.   This got totally past the VP of engineering and the CTO.    (They were just impressed that machine learning involved computing a first order derivative—something neither had since since college calculus)

So here, he knew some basic methods, but did not really know the most important ideas in the field, the important developments, how to really code this. It is clear that he had never done anything like this in his former work, nor did he really understand numerical methods.

This means that his solution would never work in production and — more importantly — that he would have no idea how to evaluate it or how to fix it. I see this a lot.   Also, he did not know the available open source codes, how they worked internally, and which one to use, or how to evaluate their performance. For being a PhD from MIT, this was unacceptable to me.

There was also a code evaluation. For me, one needs to know what runs fast and what does not. What good is a method that only runs on 300 data points?!     In this case, this interviewee had written his own javascript matrix library. Did he know the BLAS libraries and how they work? Or an alternative?     This is critical because you can’t run anything in production if the code is too slow.   I see the same problem with most ruby coders—they just don’t know numerical computing.

I was not looking to evaluate 10,000 of complex code , whether he used Agile or Unit Testing. Nor did I care about solving some high school brain teaser. I just wanted to see a small piece of code, with good engineering choices ,   a good understanding of the math, and how to make this solution work in a modest production environment .

Id rather see old fashioned spectral clustering with a Fortran library, which can scale, as opposed to trying to use a “fancy” method like NMF or LDA if you can not get it to work in production at scale. (I’m not saying they don’t scale—I am saying you better know how to get them to scale if you choose to use them)

In another interview, again a PhD (Ukrainian I think) who was very bright and had solved some good problems and had experience. He was using an off-the-shelf SVM tool—a tool I know very well.   I asked a very basic question—how do you adjust the cost parameter for the SVM regularizer.   I rephrased the question a couple of different ways to give him a chance.    FAIL   In other words, did you read the documentation of the tool and did you understand which parameters to tweak and which ones to leave at the default settings ( I kinda would like the person to have read the entire source code of the tool and know how it works. )   Again, this demonstrates a failure of the most basic mathematical concepts in ML — Regularization— and how they would apply in production. Tuning this parameter can increase accuracy by %10-15 (or more). Again, just simple stuff—but important stuff This also shows a lack of attention to notice the important details of the work. We actually offered this guy the job and he asked for a salary way out of the ballpark. If he had not missed this critical question he might have been able to make the case for the salary.

Having shared all this, I would add that I think , for you, the market is very good and you will probably not encounter anything like this. Why? All you need to do is know more machine learning than the VP and the CTO—and here the bar is very low.   Everyone and his brother has a funding to do machine learning and they usually just need to solve one small problem and get the product out the door. Most (i.e 7/10 ) CTOs and VPs know absolutely nothing about even basic   machine learning so they have no clue even what to ask. (Newton Raphson will blow them away, and they will think you are too expensive if you try compare stochastic gradient descent to interior point methods) They got their start up funded based on the market potential of the idea, and they are expected to hire people to invent their IP.

(Obviously if you are interviewing at Google or Lockheed Martin, disregard all of this and hire me once you get in)

P.S.   I was asked once by some VP/CTO evaluating me what the volume of a rectangular prism is. AlI could think of was this old Pink Floyd album Dark Side of the Moon with the Prism on it

http://en.wikipedia.org/wiki/The_Dark_Side_of_the_Moon

I would never ask this kind of question but you will probably get asked many   puzzle questions like this if you are fresh out of school (or an old man like me I guess) I seem to recall there are books and/or web sites with tons of these.

Good Luck

View Answer on Quora

What is the purpose of each folder in the root of an Ubuntu installation?

Wed, 20 Feb 2013 19:41:12 -0500

Answer by Juan Manavella:

This is not Ubuntu specific. All Linux distributions shares the same directory tree among them:

View Answer on Quora

I want to learn memoization. Can you give me some links with problems from spoj/topcoder/codeforces?

Sun, 17 Feb 2013 23:57:46 -0500

Answer by Laurentiu Cristian Ion:

Your intuition is good.

Memoization is much easier than the standard Dynamic Programming with iteration.

The hardest part in solving a DP problem (with or without memoization) is to figure out the states (overlapping subproblems) and the recurrent formula for a particular problem. Once you figured this out it is just a matter of writing a few lines of code.

A great tutorial on DP: http://community.topcoder.com/tc…

Now there are two ways to solve DP problems:

bottom-up or iteration, which is when you first compute the smaller states and solve larger and larger states based on the previously computed subproblems until you solve the entire problem

top-down or recursion, which is where memoization comes in. You call a recursive function which solves the smaller states and uses them to solve the whole problem, and in order to be efficient and not compute already computed states, it keeps every answer in a table, array or map.

At first, depending on your desire, you might like the memoization approach better because it might be easier to understand for you, but after you get the hang of it the bottom-up approach is shorter and even faster sometimes (and you don’t have to worry about the size of the stack — which may be limited on some online judges).

How do you get good at figuring out Dynamic Programming solutions?
Practice. A lot. Start with Div 2 problems.

Topcoder has, in my opinion, the most extensive collection of DP problems, and they are easy to sort by category, tags, and difficulty: TopCoder Statistics - Problem Archive (topcoder DP problems archive).

Moreover, the topcoder practice rooms help you become a faster and better coder, without the need of much debugging.

You can solve these using either memoization or the bottom-up approach. See what’s best for you.

Codeforces also has an easy way to get just the type of problems you want. You just go to the Problemset - Codeforces and click on the little blue arrow on the top right and write dp.

There are also a lot of interesting problems on SPOJ, but since they don’t do such a good job at tagging problems, there is a list here: TopCoder Forums

Nice collection of DP problems from many many sites like spoj, uva, timus, codeforces etc: http://ahmed-aly.com/Category.js…

DP tutorials:

DP tutorial on Codechef.com
Dynamic Programming Archives - GeeksforGeeks
http://apps.topcoder.com/forums/…
http://apps.topcoder.com/forums/…
Dynamic Programming Tutorial on Codeforces
TJU 2720 - Incredible! Impossible! - nice DP solution using memoization
A very smart guy teaching Introduction to Algorithms at MIT, see the playlist for DP:
http://www.youtube.com/watch?v=IFrvgSvZA0I&list=SPUl4u3cNGP61Oq3tWYp6V_F-5jb5L2iHb
Dynamic Programming Practice Problems
Dynamic Programming: Are there any good resources or tutorials for Dynamic Programming besides TopCoder tutorial?
TopCoder: What are systematic ways to prepare for dynamic programming?

If you want to extend your algorithmic knowledge:

a list of algorithms used in programming contests - Google Translate
if you’re preparing for a tech interview at a top tier company the best resource is Gayle Laakmann McDowell’s book: Cracking the Coding Interview: 150 Programming Questions and Solutions: Gayle Laakmann McDowell: 9780984782802: Amazon.com: Books
USACO Training Program Gateway - again with a great DP chapter

View Answer on Quora

What are the Top 10 Problems in Machine Learning for 2013?

Sun, 17 Feb 2013 15:23:53 -0500

Answer by Abhishek Shivkumar:

1. Churn Prediction: Churn prediction is one of the most popular use cases for people who want to leverage machine learning. It has a large business value and benefit attached to itself specially in industries like the telecom and banking. Several challenges such as the skewed nature of the data set available and the ability to decide which models to use are going to be under a lot of debate.

2. Sentiment Analysis : A lot of decisions these days are being taken on the opinion of others. We buy a product more because it has received a positive opinion and we visit a hotel most likely because it got the best rating online. Sentiment analysis has its own challenges such as how granular can the sentiment be determined, how subjective is the sentiment and so on, and hence sentiment analysis will be a good place to attack machine learning.

3. Truth and Veracity: There is a lot said online these days and it is hard to determine what is true and what is fake. We have bots smart enough to publish content like human beings and there are social aspects attached to the ratings of various entities online. I feel machine learning will be leveraged as a big challenge to determine the veracity/truth of information online.

4. Recommendations : There is such a myriad of choices available online that it is becoming even more difficult to choose a book, restaurant or even a simple product. The ability to do smart recommendations based on the user’s context and not just based on the preferences of the crowd is going to be a great challenge because it is a great deal to understand the user’s context.

5. Online Advertisement: There is a lot of work and many start ups around the space of intelligent online advertisements, but to be able to push the right advertisement at the right time in the right way to the user needs a lot of understanding of the when to target a particular customer. Machine learning exhibits a great challenge in this space in my opinion for determining the user’s behavior online to push the correct advertisement instantly when the user really needs it.

6. News Aggregation: Plenty of news is being generated around us from various different places about a variety of topics. Yet we all have a constant thirst to consume all the news relevant to us as much as possible. How are we going to aggregate news according to the user’s preference? Does his taste vary with time? How do we learn this variation? All this is going to be a challenge for machine learning and it involves a great deal of making sense of news and articles.

7. Scalability: Data is constantly expanding in variety, velocity and volumes. Can the traditional machine learning algorithms that were developed a decade back be applied to big data? I feel they will all undergo some kind of refurbishment to be able to operate on data at large scale. Can SVMs train faster? Can it be made parallel? This is going to be a good problem to focus with the rise of big data.

8. Content Discovery/Search: There are millions of people around the world on various social networks and within enterprise. How can you discover people who share similar interests as yours and what parameters are you going to consider to measure this similarity? How do we measure similarity and can we quantify this? I feel this is a nice problem for machine learning where we will face the challenge of trying to find the needle in a haystack.

9. Intelligent Learning: For example, it is still difficult to identify a behavior in a video sequence and there has been a lot of research around this space. In my opinion, with the state of art learning algorithms, I feel one of the top problems is to be enable machines to be able to see, hear and recognize like the human brain does. This means a good problem would be to leverage machine learning algorithms to use different modes of learning to achieve a particular task, be it recognition or anything similar.

10. Machine Learning for Medicine: I feel this is probably the most interesting machine learning problem for 2013 and coming future. There are so many diseases that need our attention and a lot of human time spent in researching for their cure by analyzing a lot of symptoms. Yet, two patients with similar health problems receive different kinds of treatment and eventually different extents of cure. Can we use machine learning to understand how a patient is feeling at a particular moment and appropriate recommend the right treatment for him? I feel this will change how we are going to live and will help doctors discover a lot of new medical methodologies.

View Answer on Quora

What differentiates a good modeler from a bad one on Kaggle?

Sat, 16 Feb 2013 11:06:16 -0500

Answer by Jeong-Yoon Lee:

I’d say that a good modeler must be good at (1) feature extraction + (2) algorithm selection/ensemble + (3) algorithm optimization. Newbies often focus on the second part a lot, but experts will spend a lot more on the first part, and the third part differentiates winners from other good players.

View Answer on Quora

What are some other sites that are similar to Topcoder(Algorithm section) and Codeforces?

Sat, 16 Feb 2013 10:46:26 -0500

Answer by Jayesh Chaudhary:

In no particular order:
Topcoder - Programming Contests, Software Development, and Employment Services at TopCoder
Codeforeces - Codeforces

1. Codechef - Programming Competition,Programming Contest,Online Computer Programming
2. SPOJ - Sphere Online Judge (SPOJ)
3. UVa - UVa Online Judge - Home
4. ProjectEuler - Project Euler
5. Programming Challenges - Programming Challenges
6. ahmed-aly - Virtual Online Contests
7. TJU - TJU ACM-ICPC Online Judge
8. PJU - UNION PANAMERICANA DE JUDO
9. USACO - USACO Training Program Gateway
10. TIMUS - Timus Online Judge
11. AIZU - Programming Challenge
12. URI - URI Online Judge - Login
13. ZOJ - ZOJ :: Home
14. NTHU - NTHU Online Judge
15. Leetcode - LeetCode
16. AI Challenge - Home | AI Challenge
17. Saratov - Saratov State University :: Online Contester
18. Google code jam - Google Code Jam
19. InterviewStreet - Programming Contests - Codesprints - Interviewstreet
20. Kaggle - making data science a sport
21. Herbert - Welcome to Herbert Online Judge
22. CoderCharts - CoderCharts - Social Meets Programming
23. PKU - Welcome To PKU JudgeOnline
24. CodingBat - CodingBat
25. Programr - Programr | Learn.Code.Share
26. HackerRank - Artificial Intelligence Challenges :: AI Programming Problems and Competitions :: HackerRank
27. Al Zimmermann - Al Zimmermann’s Programming Contests

View Answer on Quora

How exactly does LinkedIn generate the "viewers of this profile also viewed" list of users?

Sat, 16 Feb 2013 09:56:07 -0500

Answer by Jay Kreps:

This is done via pretty straight-forward analysis of co-occurrence, basically good old-fashioned collaborative filtering. We don’t give away details on this kind of stuff, but I will explain enough to give the gist.

Basically when two things happen together it often indicates some degree of similarity. For example two profiles could be viewed by the same person, this indicates that they share some similarity which lead that person to look at both of these profiles. Maybe the person knows both of them, or maybe the person was searching for some trait shared by both of them and they both came up in the results. Obviously you can do this with views of any type of thing, not just profiles. Views are a great kind of co-occurrence: they are very noisy but there are lots of them. But there are many other kinds of co-occurence: a person may apply for two jobs, which indicates a kind of similarity of those jobs; or a single job may be applied for by two people, indicating some similarity of the people. Once you start thinking about co-occurrences they are everywhere: two companies followed by the same person, two products reviewed by the same person, etc.

So what we have done with this is pretty cool: we take all kinds of co-occurrences we think are relevant for all the types of things on the site (people, companies, jobs, products/services, questions, searches, groups, etc), and create a giant sparse matrix where a particular entry (i,j) is the number of co-occurrences between i and j. (okay actually we generalize this to a tensor since there is an entry for each type of co-occurrence but that makes it sounds fancier than it is). Then we run our analysis on this matrix to compute sparse pairwise similarities as an Azkaban workflow made up of a series of big-ass Hadoop jobs.

So when you start working on this, first you get this connection that all kinds of things can be viewed as “co-occurances” and all those co-occurences together can be thought of as a graph where the edges are created by the things co-occuring and this graph has a matrix form, which you can do all kinds of like decompositions on, and then you spend about a week convinced this is like AMAZING and that this could somehow lead you to a deeper truth about the relationships between ALL THINGS and telling everyone at lunch about this until they stop inviting you to lunch. Then you realize you are not smart enough to figure out exactly the deeper relationship of all things, and just go back to giving people interesting things to click on, and fortunately that part works pretty well.

And of course, what the final similarities we generate mean is a bit vague. And on their own these are great for some types of recommendation, but not good enough for all types (though it makes a good ingredient in most of them). For example if you want to recommend potential connections to someone, you can use these similarities but on their own they are not a mind-blowing professional connection recommendation. For these deeper semantic recommendations we have specific systems built for each use case on top of some shared infrastructure. The co-occurrence-based results do almost always work well for basic browsing, though; the meaning and messaging is something on the lines of “here is some other stuff like that.” The advantage of this type of approach is that it is totally generic and completely agnostic to the type of thing you are recommending.

If you have gobs of data then the whole problem is pretty simple (more measurement than prediction): you just compute some simple statistics on top of the co-occurrence data and rank the top results for each item. However, if data is sparse (meaning either you have little co-occurrence data or your number of items is huge) then your results will be really noisy and there are some tricks for borrowing strength that can help. We have found that the best trick of all is just to use more data—instead of running on a few months of data, make it scale out and work incrementally so you can run on a few years of data. For this to help you have to have a way to discount for time—recent co-occurrences are usually more important than older co-occurrences.

The are a number of other subtleties. Despite our best efforts and various algorithmic approaches we were always plagued by problems of over-recommending things with very high popularity. For example for the profiles, Barrack Obama kept showing up way too often in people’s similarity results. And this defeated all our metrics too, because, well, people just like to look at Barrack Obama. We call this the Barack Obama problem.

This has been a fun project to work on. I think the first version of this at LinkedIn for people was made by Steve Stegman as a 50 line SQL script. I worked on re-writing the early version, to move it to Hadoop, increase the data used, did some de-sparsification, and make it work across the types of entities (not just people). Sam Shah and Lili Wu later did a pass on it and made it significantly more sophisticated, incremental and about 10x faster. It has been a fun project because for 7 types of things on the site we compute this for, there are 49 types of similarity recommendations that come out. So it is kind of a product factory. Every month or so someone comes by and asks if we could do a version of “that recommendation thing” for showing Xs to Ys and we say yes, but of course it will be hard and take lots of work. A few weeks later we change the job config and tell them it is done. This is always fun.

View Answer on Quora

What is the best way to find the most similar sentence?

Sat, 16 Feb 2013 09:52:17 -0500

Answer by Yuval Feinstein:

Take a look at the shingle/minhash method, which basically:
Represents each sentence as a set of shingles (character n-grams).
Calculates min-hashing - several hash functions creating a signature of each shingle set.
Groups together sets of sentences having the same signature.
The method is described in the free book “Mining of Massive Data Sets” by Anand Rajaraman and Jeff Ullman:
http://infolab.stanford.edu/~ullman/mmds.html

View Answer on Quora

What does an operating system do in the background when you try to open a file?

Fri, 08 Feb 2013 14:04:56 -0500

Answer by Robert Love:

This is the sort of question one can go deeper and deeper into. Each of the bullets I list below has a dozen bullets between it and the subsequent bullet. Nonetheless, here are the basics that just about any operating system performs when an application asks the kernel to open a file:
The application issues a system call with a name such as open, providing a path to a file and (perhaps) a requested access modes such as read or write.
The process traps into the kernel.
The kernel begins path name lookup on the requested file, which consists of walking the filesystem hierarchy and resolving each component in the path. For example, if you requested
/foo/bar/baz.txt
the filesystem first looks up
/
, then
foo
, and then
bar
. Finally, inside of
/foo/bar
, it tries to find the file
baz.txt
. At any stage, if a path component does not exist, the system call aborts, returning an error to user-space.
The kernel compares the process’s privileges with those of the file (and perhaps with any of the components resolved in the previous step). Does the process have permission to access this file in the mode requested? If not, the system call aborts, returning error.
Given sufficient access to the file, the kernel now needs to obtain a reference to the actual file, in a manner it understands, beyond the human-readable name. This differs per filesystem. On Unix-style filesystems,
baz.txt
is essentially just an entry in a table that maps human-readable names to inode numbers. Inodes are on-disk data structures that contain all of the metadata about a file, as well as links to the actual disk blocks that contain the file’s data.
With the inode number (or your filesystem’s equivalent), the kernel obtains a reference to the inode itself. The file is now “open.” In a modern kernel with a VFS, this reference includes an object that maps standardized operations such as read file or write file to the filesystem’s implementation.
The reference to the inode is stored in a per-process table, which is called something like the open file table or file array depending on the operating system. The process manages the file via a handle or index into this table; on Unix-like systems, the index into the table is a simple integer known as a file descriptor.
The kernel returns the handle to the process.
The process then uses this handle to subsequently access the file.
That’s the gist of it, anyhow.

View Answer on Quora

Is it worth it to take a level or two lower job at Facebook at a decent pay cut to be there while the company dominates/connects the web?

Fri, 08 Feb 2013 13:55:43 -0500

Answer by Chuck Goolsbee:

I was a VP of Tech Ops at a small, regional colo provider. It was a ton of fun to build a company from near-nothing, to a well-respected and mildly well-known player in the industry. In 2006, and again in 2008 Google tried to recruit me away to their Datacenters - for a lot of reasons irrelevant to your question I stayed where I was - BUT I did have to ask myself the very same question.

In 2010 Facebook called me - and even *they* asked me that question. It came up in almost every interview: “are you going to be OK going from VP to a Datacenter Lead?”

Honestly, it is kind of a silly question, because I answered it with: “Today I’m skating first line on the Chilliwack Bruins, and the Vancouver Canucks just invited me to tryout to be on their third line.”

It was a simple choice really. Go to the tryout. If they make an offer, and you like the people you’ll be playing with… pack your bag!

Bottom Line: TITLE is irrelevant. Impact is everything.

At my old job I was the top dog at a small player with impact for a few hundred customers. Today, my work has impact on one/sixth of humanity. I work in what is the absolute leading edge of technology in my sector - a model of what the rest of the datacenter industry will be doing 15 years from now. best of all, it is open, and we’ve shared the specs from the servers to the building walls.

So ask yourself: Titles? Levels? Those are just hooks to hang ego upon. Pay? That is up to you to negotiate. The core issue is: what do you want to get done?

View Answer on Quora

What are good starting points (books, tools ) for learning text mining?

Thu, 07 Feb 2013 13:16:32 -0500

Answer by Vineet Yadav:

Introduction of information retrieval(http://nlp.stanford.edu/IR-book/)

Mining the web(http://www.amazon.com/Mining-Web-Discovering-Knowledge-Hypertext/dp/1558607544)

Programming collective intelligence(http://shop.oreilly.com/product/9780596529321.do)
Statistical natural language processing(http://nlp.stanford.edu/fsnlp/)
Tools and resources
NLTK:- Nltk book(http://nltk.googlecode.com/svn/trunk/doc/book/ch02.html) and nltk cookbook(http://www.packtpub.com/python-text-processing-nltk-20-cookbook/book)

Mahout:- Mahout in action(http://manning.com/owen/)
GATE:- Text Processing with GATE(http://www.amazon.co.uk/gp/product/0956599311/) and GATE documentation(http://gate.ac.uk/documentation.html)
Support vector machine(http://www.cs.cornell.edu/People/tj/svmtcatbook/)

View Answer on Quora