Category Archives: Organizing Systems

Interview Questions: Types of Coding and Algorithm Questions

Machine Learning, Organizing Systems, Programming, Software Development

Part of a continuing series on Interviews and Interview Questions.

Today, we’re going to look at types of coding and algorithm questions. As discussed before, these can be divided up into ‘Problem Solving’ and ‘Knowledge’ questions.

As mentioned before, ‘Knowledge’ questions are very close to ‘human glossary’ questions. ‘What is the Big-O order of QuickSort? Average case? Worst case?’.

But there are some questions which straddle the line between knowledge and problem solving, answers that few but an expert in that topic would be able to exactly recall, like ‘what exactly happens between when you type google.com into your browser and the page appears?’, or ‘compare and contrast various sorting algorithms’.

For those questions, you have to be as widely read as possible, they tend to select for those who are more naturally inquisitive for things outside their specific area of expertise.

Now, for coding questions. There seem to be a few different types, which I’ll try to separate out by data structure[1]:

Arrays and Strings – Any data structure where any element is addressable in O(1) time, where elements are allocated together in memory.

Linked Lists, Stacks, and Queues – Data structures in linear form, where elements far away from the origin are O(N) difficult to access.

Trees – Data structures arranged in a tree form, with a clear root and directionality. Often sorted.

Graphs – Data structures with nodes and edges, where the edges can connect the nodes in arbitrary ways. Home to at least the plurality of the known NP-Complete problems. Note that Graph problems are a superset of the above.

Search and Optimization – Problems where you’re trying to find an optimal (or close to optimal) solution in a large multidimensional tensor or vector field. Many in this category are easily mappable to Graph Theory questions, but many are not, such as 3-D Protein Structure Prediction. Most interviews would likely not ask questions in this category, at least not very complex ones.

Machine Learning and Statistics – Somewhat related to Search and Optimization, problems dealing with how one trains a computer to solve a problem. Likely to become more and more important.

Hashes – Data structures where space is traded for speed. Generally assumed to have 0(1) insertion and retrieval

[1]Hat tip: developer.com

Interview Questions: Other

Organizing Systems, Programming

In previous posts, I’ve talked about the most important types of interview questions:

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Technical’ questions ask ‘Solve this defined problem for me.’

Today, I’ll cover some other types of questions that are known to not have much predictive power, but people still ask, either as an ice breaker, or because they have other reasons for asking these questions.

‘Ice Breaker’ questions ask ‘tell me a story about yourself, to help relax you.’

The purpose of ‘Ice Breaker’ questions is to get the conversational flow started. My personal favourite is ‘tell me about the project you’re most proud of’, because it will help to relax the candidate, and has the dual purpose of showing what a candidate is like when they’re excited about something.

‘Dumb’ questions ask things outside the normal boundaries of a standard interview.

From the link, examples might include “What kind of animal would you like to be?” or “What color best describes you?[1]” The ostensible purpose is to try to get beyond pre-programmed/rehearsed answers, looking for original thoughts. (I tend to prefer the ‘tell me what you’re most proud of’ type of question, as if you’re trying to knock a person off their rehearsed interview game, if they’re nervous, that might torpedo them, and you’re torpedoing them based on their interview skills, rather than actual skills. Better to choose a topic they know, and explore the limits of their thinking there.)

‘Illegal’ questions ask ‘I want to discriminate against you, in some illegal way’

Which questions are illegal will vary by jurisdiction, but generally include questions about things such as gender, age, marital status, religion, etc… Larger and governmental organizations tend to be better at not asking such questions, whether because of visibility or lawsuits. Knowing how to answer such questions can be tricky, because of the power differential between interviewer and interviewee, but especially because the organizations asking such questions may be hiring from a labour pool with few options.

‘Brainteaser’ or ‘Fermi‘ questions ask ‘How many piano tuners are there in New York’?

These questions are the stereotypical ‘Google interview’ question, which is funny, because Google no longer asks this type of question[2]. I happen to enjoy this type of question, and they can be very useful for back-of-the-envelope estimation, but don’t really have a useful place in job interviews.

Next time, we’ll go more in dept about specific types of technical questions. Stay tuned!

[1]My favourite story on this topic comes from the brainstorming exercise: “List all the things you could do with this brick.” People would come up with some small number of ideas (like <10) for how to use the brick. Then the facilitator would say something like: "List me all the ways that your wackiest friend could use this brick." Interestingly, this generally elicits many more ideas, as it removes some of the social opprobrium of being 'weird'. [2]cf. The British Empire no longer uses the ‘Imperial’ system.

Interview Questions: Technical

Organizing Systems, Programming

I’ve been writing about interview questions recently, most recently about ‘behavioural’ and ‘Situational’ questions. If you recall:

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Technical’ questions ask ‘Solve this defined problem for me.’

Today, I want to talk about ‘Technical’ questions. This includes two types:

‘Problem Solving’ questions, where the interviewer asks a technical question, and expects you to go through some process to solve it, similar in some way to what one would do in a job in the field.

‘Knowledge’ questions, where the interviewer asks specific questions about your field of study or work. For a programming job, they might be about memory management or data structures, for HR, they might be about what is legal or accepted practice in the jurisdiction in question, etc…

(Note that these generally don’t include questions about a resume, which I would group under the ‘Behavioural’ umbrella, as the interviewee is expected to tell a story about them.)

So what is an interviewer looking for in these questions?

For both of these questions, the interviewer is looking for command of the subject matter and problem solving ability. There’s a whole smear of possible questions between these two extremes. (‘What is an array’ to ‘Design LinkedIn’.)

For basic knowledge questions, it would probably suffice to re-read a textbook, or read (and understand!) a glossary of the topics one would be interviewed in.

For ‘Problem Solving’ questions, answers are generally more involved.

Generally, the interviewee is given a problem statement:

“Write a program which counts from 1 to 100, and outputs ‘Fizz’ when the number is a multiple of 3 and ‘Buzz’ when the number is a multiple of 5.”

This problem statement may or may not be well defined, so it falls on the interviewee to ask questions until it is adequately defined:

“Does it also print the number when it is a multiple of 3 or 5?” “Is proper syntax required?” “What language?”

(This also makes sure that the interviewer and the interviewee are on the same page.)

I like to draw a large diagram, and/or write down my assumptions in the upper-left corner when doing problems like this. Makes things explicit, people can see what you’re thinking.

One of my best bosses described his best programmer as ‘having a reason for every single line of code’. Talking through one’s code as it’s being written can help with this.

So:

Write down assumptions
Draw a big diagram
State the overall algorithm
Write down the solution, while talking about it
Think about corner cases, run an example through in your head.

Next time, we’ll talk some other types of questions, the kinds that are known to be not as predictive, but that interviewers still ask anyways, for various reasons. Stay tuned!

Interview Questions: Behavioural and Situational

Agile, Organizing Systems, Software Development, Thoughts on Thoughts

Balancing factors. Persuading people.

Yesterday, I talked about three types of interview questions:

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Technical’ questions ask ‘Solve this defined problem for me.’

Today, I want to talk about the ‘Behavioural’ and ‘Situational’ questions.

First, how are these questions similar?

Both of these are asking you to describe a solution to a problem, a problem from a surprisingly narrow set of options.

Two (or more) factors that you need to balance[1].

Helping people work together when they disagree[2].

The two factors might be technical, like how you would balance ‘Reliability’ and ‘Performance’, or they might be human, like Legal disagreeing with Marketing. Already, you can see these options blurring together. Really, these questions are really asking about how you balance things and make decisions.

The follow-up is often ‘So, once you made this decision, how did you implement it? How did you convince people that this was the correct route?’

Balancing factors. Persuading people.

So, how are these questions different?

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Behavioural’ questions ask you to tell a story about something you’ve done. You want to look at all the things you’ve done (especially everything you put on your resume), and think about what kinds of problems each of them were. What factors were you balancing there? How did you persuade people to work together and solve the problem?

An interviewer may ask you this question from a number of different directions. It’s up to you to fit one of your stories into the narrative question they’ve created (or to rephrase it such that your story fits).

‘Situational’ questions ask you how you would solve a hypothetical situation. An interviewer would present a situation with multiple factors to balance, where people disagree, and you would need to mediate, make a decision, get buy-in for your decision.

In this case, it again can be calling on your experience, but be careful that you’re actually trying to solve the problem presented, not a different problem that you’ve encountered before[3].

Summing up:

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.
‘Situational’ questions ask ‘Given this situation, how would you solve it?’

In both of these:

Balance factors.
Persuade people.

To answer:

‘Behavioural’, fit a story from your past into the question.
‘Situational’, put yourself into their story, and tell them how you would resolve it.

Next time, we’ll look at the more ‘technical’ side of interviews. Stay tuned!

[1]If you didn’t need to balance between two things, you would just choose one of them, and really, no decision is required.

[2]If people are in agreement, what decision is required?

[3]Or something from your childhood, natch.

Types of Interview Questions

Organizing Systems

Interviews. Almost everyone has been through one (or many), on one side of the table or the other[1].

Interestingly, research has been coming out saying that standardized procedures and checklists help in interviews as much as they checklists help in surgery.

To make a standardized procedure or checklist, it helps to have a list of the types of things one can ask a candidate.

A number of people have made lists of types of interview questions.

Google says: “We achieve that goal by doing what the science says: combining behavioral and situational structured interviews with assessments of cognitive ability, conscientiousness, and leadership.”

We’ll start with the first two (‘Leadership’ is best folded into these): Behavioural and Situational questions, generally considered to have the most predictive power at identifying better long-term work performance.

They’re actually pretty similar. In both, the interviewer is asking for a description of a solution to a problem (sometimes a tech problem, often a people or people/tech problem).

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

Fundamentally, answers to questions like these (are supposed to) show how a candidate defines a problem, finds root cause, and solves it, within the constraints. (One could also see how quick a candidate is on their feet, or how rehearsed, by seeing the difference in speed to how they answer these two types of questions.)

Good sources for questions like these: Any project where you’ve had to solve a technical problem (tech), any project with multiple stakeholders (tech/people), working with another team (people).

How do you answer them? Probably the best way is to go back and think about each line item on your resume in detail (Most of the lines on your resume are problems you’ve solved, right?[2]). For each of those problems, you had to work with others, define a problem, come up with a solution, get buy-in, implement a solution, some, all, or some of none of these things.

‘Cognitive Ability’ in the strictest sense of the word is most often associated with standardized tests. When combined with ‘Conscientiousness’, you get technical problem solving questions, also known as ‘Coding Interview’ to its friends.

So, really, we’re left with three types of questions:

‘Behavioural’ questions ask ‘Describe a time when you encountered a problem like this’.

‘Situational’ questions ask ‘Given this situation, how would you solve it?’

‘Technical’ questions ask ‘Solve this defined problem for me.’

Next time, we’ll go a little more in depth into these categories. If you want a specific category first, comment below!

[1]This feels very adversarial. Also, very binary. Why are there only two sides to this table? (Somewhat related, in Europe, corporate-labour relations are a trinary system, with the government being an equal partner.)

[2]Much more powerful, this is.

Why are *you* Hiring?

Organizing Systems

When you are trying to hire someone, what are you trying to do? Assuming you’re doing the sensible ‘hire once you can afford it'[1], parts of your organization are probably bursting at the seams, or people are probably feeling a bit (or a lot) overworked. Maybe a key person just left. Either way….

…you probably want to get someone in quickly, to fill the gap that you see in your organization, to take the load off yourself or the people around you.

At the same time, you want to get the right person in, someone who will add to your organization, someone who, over the time they’re with the organization, will be a net benefit.

So, ideally, you want the ‘right’ person, or the ‘best’ person quickly. But the world is not ideal, so you’ll probably have to compromise on ‘best’ slowly, or ‘good enough’ ‘as quick as you can'[5].

The issue here is that different people have different opinions of ‘good enough’, and ‘fast enough’. Sometimes, that is fine:

There’s probably a less stringent application process for washing dishes at a fast food restaurant[6], than for writing the code involved in launching nuclear weapons[7]. If you’re down one poll clerk with 30 minutes to go before the election starts, you probably need to act more quickly than if you’re a 20,000 person organization with 200 current job openings.

Next time, we’ll talk about different opinions of ‘good enough’ and ‘fast enough’ for hiring, and some things you can do about it. In the meantime:

Why are *you* hiring? What are your time horizons? Your short- and long-term goals?

[1]Aside from the nods I’ll get from most of the audience, this makes sense from a competitive perspective. In a perfect/ideal/rollingsphere[2] market situation, where different market participants have no edge beyond how hard they work, those who work harder (however that is defined). This means that whoever is working hardest is potentially (not necessarily) the most profitable and most able to hire new people, to start the cycle all over again.

[2]”Picture the market as a perfect rolling sphere.”[3]

[3]The punchline for many jokes about physicists begins with the physicist making an assumption that some object or process acts as ‘a perfect rolling sphere’. This object or process is typically nothing like a sphere, such as a horse or a bank. However, this can be a powerful tool to get started on a solution, and who knows? Maybe parts of the ‘perfect rolling sphere’ solution may be pretty close to correct![4]

[4]’Spherical chickens’ is not too far off from correct for a number of applications.

[5]If you’re Valve, or Google, this may not apply as strongly. Valve insists on ‘best’, regardless of speed, but they probably get a number and quality of applicants that would stagger any recruiter. Google probably doesn’t have to wait so much, but has a different problem of sorting through so many applications, with some interesting solutions [Wired paywall]

[6]I mention this because I once helped a nice young man with his application, so it springs to mind.

[7]I mention this because the very concept terrifies me, and I hope someone very very careful is the one doing this.

Mastery and Starting Anew

Organizing Systems, Software Development, Thoughts on Thoughts

Earlier, I had written about the new flows and new structures which confront you whenever you change jobs or organizations.

Today, I was reading an article about feelings of ‘mastery’ (and actual ‘mastery’), and how they vanish when you change careers.[1]

Interestingly, it ended up becoming a story about transferable skills, and how you develop blind spots as you become entrenched in your 10-year or 20-year (or more) career.

For me, I discovered that I had learned how to look at a software project, see the flow, find the risks, and then work to route around and/or mitigate them. Perhaps more relevantly, I found this was an uncommon skill. Interestingly, being new and not knowing the project in as much detail probably helps me more easily see the bigger picture.

I haven’t noticed any new blind spots, but I’ll continue to watch. (I also try to ferret them out whenever I see them in myself, but I have no idea how well I do at that.)

In the article, the author says:

”
In radio, information is not your goal. Someone can talk and talk and talk, but unless they talk in the right way the tape is useless to you. If they are distracted, or overly theatrical, it won’t work. (That was the problem with the first oil guy we interviewed: he was always putting on a show.) The aim is to get them to relive all the emotions they felt at the time, which will translate in their voice. This can be achieved only if you are patient and open, and take the time to establish a real connection.
”

about how she learned to give people space in an interview, to help them stop ‘putting on a show’ and actually express their inner emotions.

For me, this quote best captures why I think changing things up can be such a powerful tool:

”
Then there’s the larger matter of how you practice. In “So Good They Can’t Ignore You,” author Cal Newport says that what makes ridiculously successful people so successful is they’re experts at practicing — they can push themselves to the exact limit of their skillset and thus expand their abilities day after day. If you’re not expanding yourself in such a fashion — called deliberate practice in the org psych lit — you’ll never be ridiculously successful.
”

[1]For those who have not heard of the ‘10,000 hour rule’, it’s from Malcom Gladwell’s book ‘Outliers’. There are also some who dispute the magnitude of the importance of the rule.

New Job, New Flows, New Structures

Agile, Idle Speculation, Organizing Systems, Software Development

So you’re moving to a new organization. You know that what you will be doing will be different, you may even have read up on it.

But there are going to be all kinds of unexpected differences, and they’re likely going to come from the most unexpected directions.

Each organization has its own flows (I’m going to talk about software, as those are the flows I know best).

When an organization is writing software, there is generally some sort of version control-coding-testing-release pipeline. However, there are many different pieces of helper software for each of these steps.

There’s the flow of information as clients are using software. Information supplied by the clients, information supplied by your organization, and these all have to work together smoothly to solve whatever the client’s problem is.

The way that feedback from clients is turned into actionable items can be vastly different between organizations.

On the transition line between flow and structure, the way that teams are divided often reflect historical decisions made in antiquity, often the division of labour of the first few people writing the code for the first iteration of the product.

Information will be flowing through this system while the software is running, effectively handing off from team to team.

There will be structured and unstructured information flows in the organization. Many people are at their most effective when they receive all of their new information before it arrives through official channels.

Even the structured flow of information can be very different, for example in a very flat organization such as Valve.

So, be mindful, and watch for the different flows. You may be surprised at how different each organization is (or how similar).

What do Numerical Software Development Estimates Actually Mean?

Agile, Organizing Systems, Software Development

What do numerical software development estimates actually mean?

What do they mean for you?

(I’m talking especially about team-based estimation, such as that in ‘planning poker‘, but I’m guessing whatever conclusions we may have would hold for other methodologies.)

I see the general objective here as coming up with a number for each task, and a number for how much your team can typically do in an amount of time, such that these numbers are reasonably fungible.

Traditionally, estimates would be given in ‘programmer days’, or ‘wall clock time’, depending on whether you had read ‘The Mythical Man-Month‘ or not[1].

More recently, there has been a back-and-forth between ‘amounts of time’ and some sort of dimensionless unit called ‘complexity points’.

Various teams that I had been a part of struggled with ‘complexity points’. In their strictest definition, something which was simple and repetitive would be worth few ‘complexity points’, even though it would take many hours of some attention or nursemaiding to finish the task.

Strict ‘amounts of time’ are no better, because each person does each task at a different rate.

We had the most success with ‘relative complexity’, or taking some small and large tasks, assigning them numbers, then rating each of the other tasks with respect to these goalposts.

Even this has its limitiations, though. Fundamentally, they question you’re asking when you’re deciding to put something into a sprint is ‘can we still accomplish everything if we include this?’. Because of limiting reagents (specific people who are bottlenecks for many tasks) and interdependence between tasks, this can be problematic. The standard way of getting around this is to insist that all tasks are independent and small.

This worked reasonably well, it’s just that sometimes you need to rewrite or refactor an entire application.

What are your experiences? How have you dealt with this question? How many points would it be worth to research and present on this topic?

(This post came out of a fb conversation with D about what estimation numbers mean, and have meant at various times.)

[1]One of the upshots of this is an observation made by someone at work (I think F) which was that Gantt Charts are excellent for deriving dependencies, but terrible for estimation.

New Divisions of Five Management Roles

Agile, Idle Speculation, Organizing Systems, Software Development

Yesterday, we talked about five management roles:

Performance Manager (Worker Evaluation)
Estimatrix (Estimator)
Product Owner (Prioritization)
Scrum Master (Removing Obstacles)
(People) Development Manager (Development Conversations)

In a traditional corporate structure, these five roles are combined in one person (your boss).

However, there are many ways to divide these roles, and many reasons to do so (the simplest being that different people are good at different things).

Valve famously has an incredibly flat structure, where each person has a set of peers (the rest of the company) who handle performance management, and all of the rest of the roles are performed by each person themselves. As they say, occasionally teams will form with people splitting off into roles, but that’s all dynamically allocated by the people involved.

Your standard ‘Scrum‘ Agile shop will tend to put the ‘Performance Management’ and ‘Development Management’ into a ‘People Manager’. ‘Estimation’ is done by the team as a whole, the ‘Scrum Master’ or ‘Obstacle Remover’ is traditionally not the people manager, but is a separate role. The ‘Product Owner’ can be the ‘People Manager’, or someone else, sometimes an external product or project manager, but is generally not the same person as the ‘Scrum Master’.

I would argue that this tension between prioritization and removing obstacles is one of the reasons the system works better than many.

There seems to be a growing trend to separate Performance from Development[1], with some companies having separate reviews in different parts of the year for each of these. This can be especially helpful as many people are unlikely to be relaxed enough to think about how to take beneficial risks in the future when they’re tied up in knots about whether their boss wants to fire them.

I think it might make sense to push this to its logical conclusion, and have separate people for these separate roles in a company. The ‘Development’ role feels almost like a traditional HR thing, but I feel like to best serve employees, it would really need to be a separate department, called ’employee growth’ or something similar.

What do you think? What have you heard about how different organizations split these roles? How do you think they should be split?

[1]Development as in ‘where is your career going?’