Monday, September 28, 2015

Node.js and MEAN stack tutorial reviewed

Staying on top of tech trends is part of our jobs.  The days of just knowing say... COBOL and that is all you know are long gone.  If you are not expanding your skill set through regular if not daily learning you a falling behind.  Everyone has there own method of learning and you should find what works best for you.  If it is reading books then check out https://www.safaribooksonline.com/. If it is taking online classes there are sites like https://www.udemy.com/.  Do what works for you but always be learning because not only it is cool learn new stuff but your career as a programmer requires it.

...Rant about continuous education over...

First off I am not a Node.js rock star, but I know the power of Node.js and the rising popularity of it.  Being able to say I was a master of the JavaScript and Node.js universe would would be amazing.  Which is why I learn about it even though my day job doesn't have anything to do with node.  Although I do like to throw in some Node.js from time to time for utility applications.

When you see charts like this on Indeed.com there is no denying that there is demand for people who learn Node.js:




So along comes this Node.js Tutorial which I of course have to read and follow:
https://blog.udemy.com/node-js-tutorial/


The author, Moshfegh, clearly knows he stuff and is passionate about helping people join him in the Node.js ecosystem.  He doesn't just write about Node.js but also covers the entire MEAN stack. (MongoDB Express.js Angular.js Node.js)  Before Node.js JavaScript only executed in browsers but then in 2009 that barrier was shattered.  With JavaScript on the back-end and the front-end engineers can become experts in one language and create quality code that follows the same standards on both tiers.  While I am tempted to go more into the wonders of Node.js and the MEAN stack I must defer to the tutorial because he is better at it then I am.  Hopefully with his tutorial and a little practice I will be able to teach others.  The blog post is just the beginning.  The author is creating his own course on https://www.udemy.com/ to expand upon the material he was able to cover in a blog post.  So if you enjoyed the post like I did you will be able to learn more with a course of the same level of quality.


Happy Learning.

Friday, September 25, 2015

Programmer turned Home DIY Enthusiast

Programmers are smart, educated, and logical so hammering a couple nails should be no problem for us.  That was part of my mindset when I took on a massive home renovation this summer.  We were running out of space for our growing family so we needed to expand.  The plan was to gut the second floor and reclaim some storage space and in the process add a third bedroom on that floor.  I know I didn't have time to it all myself so there would be subcontractors hired but I was convinced I wouldn't need a general contractor because I could take care of that myself.  After all I was smart guy and had done a few projects around the house... I could do it.

Being your own general contractor is like starting a new job as a project manager in a field you just have a vague idea about. The project you are assigned to is also one of the most important in the history of the company so there is little room for error.  That is what it is like being your own general contractor on the side.  Oh did I mention that being your own general contractor also means you have to get your hands dirty doing the parts of the project you don't have "staff" to fill?

So what is a savvy coder to do when faced with such a challenge?  As a programmer what do you do when you have a vague idea what to do but are not sure of the details? Google. 

What I hired people to do:
  • demo existing space
  • frame the dormer
  • put the roof on
  • put siding on
  • electrical
  • plumbing for heat
  • carpet
What I did:
  • insulation
  • drywall
  • sub floor
  • paint
  • trim
  • baseboards
  • window casings
  • hang doors
  • and many more odds and ends

The biggest lesson learned was to find a mentor and don't be afraid to ask questions.  There is a lot of know and frankly a lot of the techniques and tips are unwritten and you either learn the hard way by doing it wrong or you ask questions.   Just like in programming google can only get you so far.  You need to have a mentor, a proper teacher, or a quality book on the subject to understand how to do it without a lot of growing pains.

Inspections!! There are a lot of inspections, permits, and plans required with this type of a project.  The regulations and red tape can grind everything to a halt if you don't stay on time of it.   This is just like in the coding world.  Your project can be crushed if you do not know how to operate within your companies policies and know who to go to get sign off on things and overcome road blocks.

Coordination was critical to getting this project done.  Every step has several preconditions that need to be fulfilled.  So for example to hang drywall you need the drywall delivered, rent a drywall jack, insulation inspection complete, rough electrical done, rough plumbing done, and stuff like that.  Just like in the software project management world if one precondition isn't fulfilled things grind to a halt.

I cannot express enough how great of a learning experience this was not just as a homeowner but as a project manager.  The struggles were real and stressful but in those struggles there was personal growth.  Next time I don't know if I would rather hire someone to do it but this time I was glad I did it.  So if you are looking to grow your skill set for your career look around you and maybe you can find something outside your field to help you like volunteering, joining a club, or side projects even including home improvements. 

Read More about Soft Skills:

Friday, September 11, 2015

If I only knew Javascript...

If I only knew JavaScript what a wonderful world it would be.



Created by Daniele Rossi


Now do not get me wrong I have written tens of thousands of lines of JavaScript code in my career but do I really know  Javascript? No. Have I ever been taught how to do JavaScript ? No, like the comic indicates like most people I learned from finding snippets that solved a need I had and modified it.  Do I think it would be incredibly valuable to be good at core Javascript? Heck Yeah!

We have seen processing move from mainframes with client terminals, to fat clients, to thin clients with the bulk of the work happening on servers, and now things are moving back to the clients.  JavaScript framework flavor of the months a dime a dozen but a lot of exciting development is going on in the JavaScript world.  Making a career focusing on JavaScript is becoming more common.


JavaScript was originally created in just 10 days.  Brendan Eich while working at Netscape was told to go create a language to do things in the browser. Obviously 10 days isn't much time so some of the syntax was borrowed from Java and several other places and originally called Mocha.  Then it was called LiveScript, then ECMAScript when it was standardized, but it eventually became called JavaScript mostly as a marketing ploy. JavaScript and Java share almost nothing in common besides a name.  However this little language hacked together in just over a week has become one of the most popular and wide reaching languages in the world. With Node.js it has even moved off of the browser and has started running server side applications for itself. 


JavaScript has a spec! BOOKMARK IT ASAP  It was just updated in June so it constantly evolving to fit our development needs.

There are many quirks in JavaScript so while it is an pretty easy language to get started in it is difficult to master. Coercion for example is something I have often struggled with and the JavaScript concept of 'falsey'.  Fundamentally coercion is like Java autoboxing when variables of different types are often not so gently coerced into an type the is equivalent enough to be compared.  It is this type flexibility that is at the same time both an asset of JavaScript's and a point of mystifying behavior.

Examples of Coercion madness:

(true == 1) => true
(true === 1) => false

or

if ('true') {
    'true' == true  // returns false
    'true' == false // returns false
}


Popular JavaScript frameworks:

Learning resources:
Code School - Fun quick online classes without a lot of depth but a great overview
You don't know JS books - Free high quality books by Kyle Simpson

Links to JavaScript posts:

Thursday, September 10, 2015

24 hour challenge

I just read this great post by John Sonmez about a 24 hour $100 challenge entrepreneurial challenge.  This is a fairly old post but the content on www.simpleprogrammer.com has rarely failed to be valuable.  It is a three part post that I was hooked on after the first couple paragraphs.

It is inspirational.  Go read it and come back to this post. I will wait.

...

... read it

...

...

... no faking



Okay so now you have read it or are just a cheating yourself but I like the idea of a challenge like this.  Do you want to try something like this? It doesn't have to be the same thing but it could be or it could be something like generate as many hits as possible in one day to an entirely new blog or hack together a side coding project.  If you are interesting is something like that comment on this post or send me an email.

Please think about it.  I can do it myself against only myself but that cuts back on the pressure that makes you sharper.

Recipe - How to start a new job poorly

So you passed the interview, negotiated a salary you are happy with, and are leaving your current position for greener pastures.  It is time to coast for the next six months or so when people stop thinking of you as the newbie.

New Job Cookbook:


Dress as comfortably as you can get away with

Dressing purely for comfort and not caring what people think of you is not professional.  Unless it would be totally out of place I recommend dressing at least as nice as the best dressed person you encountered while interviewing.  Some people may have been walking around the office in shorts and flip flops but unless that is how everyone is dressing that is not a good tone to set.  If you are not sure then error on the side of dressing up rather than down.  No one will take note if you start dressing more casually later but if randomly you start coming in a dress shirt and tie people prepare for the half joking questions asking if you have an interview later.

Find a way to prove yourself to the team and so them that is was good their good fortune to have hired you.

They offered you the job and wanted you to join the team.  You have nothing to prove. You have already proved it. Looking for a way to make a mark day one will just add stress and get in the way of learning your new work ecosystem.  Relax.  Build relationships and get a lay of the land.  Only then can you carve out our niche and add value.

Do not ask for help.  That will make it seem like it was a mistake to hire you.

You are new and just keeping the lay of the land you should be asking questions to speed up the process.  People also enjoy feeling useful and helpful.  A question can be a good icebreaker and an opportunity to set the tone for the relationship with your new colleague.  Be open and honest about what you don't know and you will be surprised how willing people are to help.

Stay in your cubicle and wait for people to see you out

To be productive at a new company you cannot be an island on to yourself.  Even if it is against your nature you need to get out there and meet people.  Do not have lunch at your desk.  Seek out opportunities to meet people.  A good way to do this is sitting in on other teams stand ups.  Being a fly on the wall for stand ups is a great way to meet people outside of your immediate team and increase the bredth of your knowledge about the company.  To be an effective member of the team people need to know you and you need to make an effort to know them.  For the first couple weeks I like to keep a text files with short bios about people to help me keep track of things and not get overwhelmed. 

There are always a few key people to know that might hold no real power or authority but are the keys to the social network of the company.  For example always be nice to the receptionist, office administrator, and cleaning staff. Just because a person has a less glamorous job than you doesn't mean their position isn't vital to the day to day activities of the company.  You would be surprised how many hurdles can be overcome just by being nice to the unsung heroes.  

Criticize how things are done and offer better ways of doing things

Before you start offering unsolicited advice and criticism learn the details as to why things are that way.  Decisions are not made a whims and likely at one time there was a reason behind the decision that someone felt was a good one.  Even if you don't agree and everything seems totally wrong hold your tongue until you learn how they got to be this way.  Once when I started a new company everything seemed to be anti-pattern and opposite of the way the industry was headed but it would have done me no good to point that out.  The people maintaining the code base built it years ago and were very proud of it.  You would not criticize someone else's child and likewise you should not criticize their code.  Learn how things are and slowly make suggestions and advise who to course correct but only after you understand how things got to this point.  Starting off criticizing the way things are done and showing people a "better" way of doing things will just make you seem arrogant. 

Never talk to your new boss unless spoken to

People are busy and more than likely your boss has more one her plate than just your onboarding.  Make a point of touching base regularly.  John Sonmez from www.simpleprogrammer.com recommends sending a weekly status report.  The status report does not have to be detailed and can just be a bulleted list of what you have done.  These weekly status reports can be a nice way to stay on your bosses radar and provide documentation for future discussions around reviews and why you might deserve a promotion.  Starting this habit week one will show your new boss that you are not a typical employee and you will make their lives better by having you on the team.

You know the technology so you don't need to understand the business to be effective

The technology you are working on is often the easy part.  There are books, blog posts, conferences, and webinars to help you acquire what ever technology you need to learn but there is rarely a book out there about your company.  If you work for an insurance company you should learn about insurance. If you work for an financial services company then you certainly need to learn about investing.  In the vast majority of cases technology is a means to an end not the product itself so you need to learn business domain you are a part of so that the technological decisions you make are in tune with the bigger picture.

Assume your org chart is an accurate reflection of reality

The org chart may tell you the reporting structure according to HR but that rarely defines how things actually are.  Take note of the people who are either friendly with those in charge and the people who teammates fear crossing.  A persons may not have a fancy title but that doesn't mean that they are not a trusted adviser.  Not all people seek to climb the corporate ladder so it is quite possible the rock stars who drive company do so happily from the shadows because they enjoy where they are at.  Most importantly know who you do not want to be mad at you.  There are often a few people around who are in everyone else's business and can make your life difficult if you cross them.  I know it isn't how it should be but there often people you just have keep a wide berth of and try to being friendly with because they will make your life miserable if you don't.



Wrap up:


Clearly this was a recipe for what not to do.  Your first week at your new company will set the tone for your tenure there. The impressions that people make of you will be difficult to change later.  Sure nothing is set in stone and you can turn things around later but a little effort can help help you start off on the right foot.  Determine how you want to be perceived and take action to present that to your colleagues.

Don't coast through your first few weeks at a company. Use that time to build a foundation for your future there. 

Thursday, September 3, 2015

Mutation Testing - 9 Lessons Learned

Mutation testing is amazing.  It still shocks me that it has not caught on despite being around for decades.  This post is in regards to what the value add that comes from doing mutation testing.

Don't know what mutation testing is? Check out these posts:
Mutation Testing 101:

A mutation is a single syntactic change to the code. Each mutant is different from the original by a single mutation. These small changes can have big results; such as changing an operator from addition to subtraction.  Then your unit tests execute against the mutant code and in theory our tests should fail because the code is wrong.

    public int additionOrSubtraction(boolean isAddition, int x, int y) {
        if (isAddition) { //mutant: replace isAddition with !isAddition
            return x + y; //mutants: replace + with -,*,/,%
        } else {
            return x - y; //mutants: replace - with +,*,/,%
        }
    }


Lesson #1: Mutation Testing improves code reviews

Do you do code reviews? I bet they focus 95% on the code that will go to production and maybe a quick peek at the tests.  Despite the fact that a huge portion of the code base are unit tests we really don't review them often due to there being a finite amount of time.  Mutation testing came to my attention when searching for a method to improve the code review process.  With mutation testing the framework adds to the metrics to measure the quality of the tests suite.  Metrics provide an unbiased way of illustrating the tests that add value and the tests that just provide a false sense of security.

Lesson #2: Not all mutations add value

Some mutations are just silly and getting 100% mutation test coverage can be more trouble than it is worth.  The most common example of this I have seen are mutations that remove log statements.  Verifying logging statements add little to no value in 99.9% of cases.  The PIT mutation engine does not differentiate between the different types of void method calls so logger calls are fair game.  These failures can be ignored and maybe the case can be made to try and exclude them from the remove method call mutator.

Lesson #3: Start small

Mutation testing is very CPU and time intensive. For example in a previous post I mentioned how a build that normally takes 30 seconds to compile and run all the unit tests took over 9 minutes with mutation testing.  The first place I start is actually in Eclipse and run a smaller test suite using PitClipse.  This way you can see if your tests even can withstand a mutant onslaught. Also the mutationUnitSize configuration point of PIT allows for the slower ramp up of the number of mutations that are right for your project:

> mvn org.pitest:pitest-maven:mutationCoverage -DmutationUnitSize=2

Lesson #4: Laziness creates surviving mutants

This lesson was learned from the same mutator can caused the example in lesson #2. Not verifying the results of void method calls causes more mutations to fail than almost anything else.  In many cases this is just due to pure laziness.  I just found a project  where all builder method calls to populate POJOs only had an assert to make sure the result was not null.  So the tests did not care what was populated in those beans and just focused on them not being null.  This is a perfect example of having 100% code coverage but the tests themselves being worse then if they did not exists at all.  Do yourself a favor and verify every element in the objects returned. I know it takes time and many asserts but that extra effort may save you from a production bug born from laziness.

Lesson #5: People won't see the value of mutation testing at first and that is okay

Mutation testing sounds interesting to people as a theoretical exercise but the benefits of it are often hard to see.  It takes a mature organization that is already dedicated to a strong unit test base to even consider adopting it.  To get it adopted you, yes you, need to be the zealous advocate for mutation testing.  Just do it.  Update your builds and put a mutation testing job in your continuous integration tool like Jenkins.  If your team hopes to do continuous delivery you need a rock solid test base.  Start pointing out mutations that result from recent commits just like you would if the commits result in bad code coverage.  Do not try to convince people of the value of mutation testing but instead show them the lack of defects that could result from weak tests.

Lesson #6: Mutation Testing is not just for Java

Although the mutation testing tool I am outspoken about is for Java that doesn't mean that there are not frameworks for other languages.

(I have not used any of the following tools and am therefore unable to attest to their value)


Lesson #7: Be relentless in advocating for mutation testing

Mutation testing has been around since 1971.  The days have long passed when the computer power is not available to make make mutation testing a viable tool.  If you don't push for it to be adopted it likely won't because we haven't reached that critical mass to give it a life of its own.  So do it and email out the reports. Do a lightning talk or brown bag session on mutation testing to get the word out.  Honestly I don't care if you even need to borrow heavily from or steal the Mutation Testing Slideshow that I recently created. Get the word out.

Lesson #8: 100% mutation kill score is possible but expensive

As previously stated some mutations are silly not really useful but there are times when there is no margin for error and your code needs to work.  Say for example you are working on code for a medical device.  There is no room for error when someones life is on the line.  You need mutation testing to do everything you can to be defect free in cases such as that. However some mutants just aren't worth the time needed to kill them.  It is similar to writing unit tests to test all getters and setters in POJOs.  Is it really worth your time to write tests for basic getter and setter methods. I don't know that is a personal decision but squashing some mutants might add even less value than those unit tests.

Lesson #9: Mutation testing makes you a better programmer

Knowing the ways your code could break and how good your tests need to be to protect against that is not a skill that is developed overnight. Mutation testing ensures that you don't cut corners and take the easy route.  Write solid code and bombproof unit tests.




Links to other posts related to Unit Testing and Mutation Testing:











Tuesday, September 1, 2015

13 steps to make TDD actually work

Why TDD doesn't work for you?

This is one of those situations where being brutally with yourself will help.  Test Driven Development (TDD) does not work because you are not doing it.  It is as simple as that.  You gave up on TDD because it was it was sounded good but it was hard.


What is TDD?

At its core TDD is thinking about your unit tests before you write the implementation code.  When you write the code first you are likely focusing on one happy path end result and coding to try to reach generate that result.  That closes your mind off to all the possible outcomes, all the possible input scenarios, and all the possible things that could go wrong.  So in TDD you think about the tests first so you do not have blinders on to scenarios beyond the happy path through your code.


What is so appealing about TDD?

When you were first introduced to TDD you were likely shown a hello world like feature then you were told how to write the tests for it and then shown you write the code to make your test pass a little at a time.  It was glorious.  The solution to better tests was just writing your tests first! Your tests could be better by thinking about everything and not just the happy path.  It is such a simple concept you can do that! No problem.  Less bugs means more time doing fun coding and less time doing support work.


So why aren't you doing TDD now?

You ran off to your project base code and tried to implement the feature using TDD and it was not as easy as that hello world example.  You tests required mock objects which you had to just guess about it because you were not sure exactly how you would implement the code.  After some struggles you either just gave up because you had a deadline and this was slow or you banged out some hard won tests and started on the "real" functional coding.  After you wrote some real code you figured out that tests you guessed about still weren't passing because the guesses you made about the mock objects and service calls were just tiny bit off so you needed to fix those to even some of your tests to pass.  How is disheartening! You spent all that time writing those tests first and they were not really adding value and you weren't "writing your tests first."  So you gave up on TDD.  It was hard. It wasn't the solution to your problems like you were told and all it did was slow you down.


Why should you give TDD another chance?

Because you were doing it the hard way and you didn't even know it.  Doing TDD is a mindset which you aren't going to develop just because you watched one presentation about it and tried it once.  You were sold on the concept but it practice it was hard.  TDD is not as black and white, not as all or nothing, as you were told.


How should you being doing TDD:

  1. Get your feature ticket from Jira, a post it note, or wherever
  2. Do not open up your IDE
  3. Think about the feature and do a quick design of it either in your head or on a whiteboard
  4. Write down some test scenarios
  5. Now open up your IDE
  6. Create your "real code" class and method signature
  7. Create your unit test class
  8. Create a unit test but instead of writing the test case just stub out the unit tests and create comments about what the test scenario you want to cover (see sample below)
    • Include both the input, expected results
  9. If possible write the assert statement for those expected results
  10. Repeat steps 8 and 9 until you have covered every scenario you can think of
  11. Go write your "real code"
    • pause occasionally if you think of a new scenario and update the unit test class to include it
  12. Now fill in the unit test to cover those test scenarios including all the inputs, mock objects, and even better assert statements
  13. Tell your coworker about how you actually did TDD and it made your code better


Sample Stubbed out Test Class

public class AdditionHelperTest {

 /**
  * test basic addition
  *
  * input: 1, 2
  * 
  * expected: output 3
  * 
  */
 @Test
 public void testAdditionHappyPath() {
  fail("Not yet implemented");
 }

 /**
  * test null input
  *
  * input: null, 2
  * 
  * expected: output illegal argument exception
  * 
  */
 @Test
 public void testAdditionNullFirstArg() {
  fail("Not yet implemented");
 }

 /**
  * test null input
  *
  * input: 1, null
  * 
  * expected: output illegal argument exception
  * 
  */
 @Test
 public void testAdditionNullSecondArg() {
  fail("Not yet implemented");
 }

 /**
  * test negative numbers addition
  *
  * input: 1, -2
  * 
  * expected: output -1
  * 
  */
 @Test
 public void testAdditionNegative() {
  fail("Not yet implemented");
 }

 /**
  * test addition large numbers
  *
  * input: Integer.MAX_VALUE, Integer.MAX_VALUE
  * 
  * expected: exception
  * 
  */
 @Test
 public void testAdditionLargeNumbers() {
  fail("Not yet implemented");
 }
}


What is next?

As you get better at thinking about TDD you may be able to create more of the test code first, for some features, and you will get faster at generating those test scenarios.  You will have less bugs in your code. You will then have more time to do fun new development and spend less time doing boring production support.

Other Testing Magic:

Friday, August 28, 2015

Big O Notation


If you are not very good at something then make yourself write about to really know it.


Big O Notation is a way of expressing the upper-bound complexity of an algorithm.  Seemingly minor changes can have major affects on an algorithms performance.  At the surface level these differences can seem insignificant but when time is of essence and every millisecond matters, say in a stock trading application, or when crunching large quantities of data those minor changes can have huge affects.  Whether your team mentions big O notation regularly, just in interviews, or never it is part of the fundamentals that can mean the difference between being a pretty good coder and a great software engineer.


If you are visual learner like me than this will speak volume about how little change can have big results:




Scalability in words not pictures

An O(1) algorithm scales the best and scales better than an O(log N) algorithm,
which scales better than an O(N) algorithm,
which scales better than an O(N log N) algorithm,
which scales better than an O(N2) algorithm,
which scales better than an O(2N) algorithm.

Examples of Big-O scalability

O(1) has no growth curve.  An O(1) algorithm's performance is independent of the size of the data set on which it operates.  So this implies a constant speed which doesn't necessarily mean fast depending on what the algorithm is doing.

O(N) says that the performance is directly related to the size of the data set.  Such as searching a linked list but since Big-O is the worst case the element you are trying to find is at the end. If someone talks about O(2N) they are talking more about the implementation details vs the growth curve.

O(N+M) is a way of saying that two data sets are being accessed and that their combined size determines performance.

O(N2) says that the algorithm's performance is proportional to the square of the data set size. This happens when the algorithm processes each element of a set, and that processing requires another pass through the set. Such as the famous bubble sort is O(N2).

O(N*M) indicates that two data sets are involved but processing each element requires the processing of the second data set. Don't get sloppy and use O(N2) if the data sets are about the same size.

O(N3) and beyond means lots of inner loops.

O(2N) means you have an algorithm with exponential time behavior. There's also O(100N), etc. In practice, you don't need to worry about scalability with exponential algorithms, since you can't scale very far unless you have a very big hardware budget.

O(log N) and O(N log N) might seem a bit scary, but they're really not. These generally mean that the algorithm deals with a data set that is iteratively partitioned, like a balanced binary tree.


Don't be scared of the Big-O

Talk of Big-O notation can be of the tactics employed by hot shot programmers who are trying to increase their credibility.  Big-O isn't something to be scared of it is just another tool and once you understand the tool you can wield it like a Jedi using her lightsaber.

Big-O isn't just about the CPU

While it is true the vast majority of time Big-O is discussed it is in regards to computing cycles, disk usage or memory consumption can also be referenced by Big-O notation. However unless otherwise stated speed, raw CPU horsepower, is what people are referring to.

Big-O embarrasses the worst case

When Big-O is discussed it is assumed you are talking about the worse case scenario of hitting the upper-bound complexity.  For example there is really no point in worrying about the CPU time it would take to sort and already sorted list but a list in reverse order is a different matter entirely.  In interviews it can't hurt to mention "worst case complexity to add to your credibility.

Big-O doesn't have time for little numbers

In Big-O you really don't care about the lower ordered values because as N gets larger enough those lower ordered terms get less significant. Constants are also irrelevant want talking about Big-O. A one time calls that takes 20 seconds for a response may seem like a big deal when the entire operation takes 23 seconds to process 300 data elements but when you are talking about 3,000,000 elements that one time 20 seconds call becomes trivial.

What is Big-O good for?

So what is all this Big-O stuff good for besides intimidating and strutting your computer science chops around? Performance.  Time is money.  If something is slower it can hurt the user experience.  If something is slower you need more servers, more powerful, or more and more powerful servers to keep the run time in a reasonable time frame. Features are generally what sells but performance keeps users.  Startups will often worry about features first and then performance but as they mature that performance concern becomes greater and greater.  Discussing and thinking about Big-O notation and the complexity of algorithms will help train your mind to think performance even when focusing on features.

Some keys to a successful Software Engineering hunting:


Monday, August 24, 2015

Riddle me this

This will be quick and it doesn't have any code but it is an exercise in thinking through a solution to a problem.



Picture this:

You have 8 balls, say ping pong balls, they all all the same weight except one heavy one. Your challenge is to determine which one is the heavy one but you only allowed to use the scale twice.







Given 3 uses of the scale the solution would be easy but with only two chances to weigh the balls you need to get creative.  It took me a while to get it but once you do it will seem so obvious.


Hint: Not all the balls need to be weighed.




example of another riddle turned into code

Build a crawler

Have you ever wondered how a web crawler worked under the hood?  Me too... so I wrote a quick and dirty one.

What does a web crawler do?  It starts at a URL and collates all links found on that page then follows those links and does the same.  This continues either indefinitely or until some type of limit has been hit. Sounds simple enough right? So go build one yourself just for fun.  If you wish to productize it go a head the world could always use another good crawler for testing purposes.  In the past I have used crawlers as a way to warm up a site after a deploy or just do a smoke test by watch logs for exceptions during a crawl.


Here is the code for the crawler I wrote over the span of a couple hours:

https://github.com/bobbylough/crawler

What I felt were the tricky bits:
  • Parsing an HTML page to find the links
    • there are libraries out there to help
    • the regex used in my code works but is not perfect
    • doesn't handle Javascript bound actions.. I would love to see the code for a crawler that can handle those
    • found links to images which really should have been ignored
  • Determine where to crawl next
    • all links discovered were put into a queue
    • this created a breath first crawling behavior
    • in retrospect I should have limited the number of links able to be discovered on a page
    • followed external links... this is often an option on crawlers but I did it by default

How does your design an implementation improve upon what I wrote?  I am super curious what people come up with after spending a couple hours creating a crawler and trying to stick to core libraries.

Monday, June 29, 2015

808: Reverse Integer / Integer Palindrome

The popularity of my Find the middle of linked list in one pass post here is another classic interview question.  Once again the goal of this question is to see if you strip away the libraries and tools from the software engineer does she still have to fundamentals needed to think through a problem.

Determine if a given integer is a palindrome. 


(think about it before you jump to see what the answer is)

.              

..

...

....

..... 

....

...

..

.

Once you complete your solution on the whiteboard ask a question or two. Such as can negative numbers be consider palindromes? Can numbers that end in zero be palindromes? (example: does 100 == 00100) Asking a question shows your understanding of the problem and the ways algorithm can be enhanced.


Java Solution:



boolean isPalindrome(int x) {
 return x == reverse(x);
}

int reverse(int x) {
 int result = 0;
 int digit = x;
 
 while (digit > 0) {
  int modulus = digit % 10;
  digit = digit / 10;
  result = result * 10 + modulus;
 }
 return result;
}


Related Content:

Wednesday, June 24, 2015

Find middle of a Linked List in one pass/loop (Pseudo code/Java/JavaScript)

This is a classic interview question that I think most engineers should basically have memorized.  Why; because it is a simple problem with a simple solution but very low level compared to our day to day activities.  Doing stuff like this regularly even as a senior engineer is important exercise for your mind.

I have been asked this question I half dozen times in job interviews.  If at some point in time in the future I am ever interviewing you, watch out of this question because I like to use it as an opener. (If you read my log and I am interviewing you then I think you should have a leg up on the competition.)


Write an algorithm to find the middle node of the linked list. With the caveat that you can only make a single pass through the list.


(think about it before you jump to see what the answer is)

.              

..

...

....

.....

....

...

..

.

Disclaimer: Some people feel this is 1.5 list traversals despite using a single loop.  This question and solution is not unique to this blog post so opinions of this are mixed. I respect that and you should form our only opinion,


The solution is straightforward once you step back and think slowly.  Have two pointers that traverse the list but one of them only moves every other time.  So by the time the faster moving pointer gets through the list your second pointer is at the middle node.


The problem I ran into was... basic linked list really aren't a part of normal libraries.  I have never attempted taking this algorithm beyond pseudo code before so it was a fun little challenge to implement as simply as possible.



Pseudo code

    Node current = LinkedListHead;
    int length = 0;
    Node middle = LinkedListHead;
  
    while(current.next() != null){
        length++;
        if(length % 2 == 0) {
            middle = middle.next();
        }
        current = current.next();
    }

    return middle;


Java Solution

public class MiddleOfList {
    
    public String findMiddleOfList() {
        LinkedListNode tail = new LinkedListNode("data5", null);
        LinkedListNode node4 = new LinkedListNode("data4", tail);
        LinkedListNode node3 = new LinkedListNode("data3", node4);
        LinkedListNode node2 = new LinkedListNode("data2", node3);
        LinkedListNode head = new LinkedListNode("data1", node2);
        
        return findMiddle(head);
    }
    
    private String findMiddle(LinkedListNode head) {
        LinkedListNode current = head;
        int length = 0;
        LinkedListNode middle = head;
      
        while(current.nextNode != null){
            length++;
            if(length % 2 == 0) {
                middle = middle.nextNode;
            }
            current = current.nextNode;
        }
        return middle.data;
    }
    
    private class LinkedListNode {
        public String data = null;
        public LinkedListNode nextNode = null;
        
        public LinkedListNode(String data, LinkedListNode nextNode) {
            this.data = data;
            this.nextNode = nextNode;
        }
    }
}


JavaScript Solution

Here is an implementation on JsFiddle.  This is an attempt to keep things as simple as possible.

var tail = {data: "data5",next: null};
var node4 = {data: "data4",next: tail};
var node3 = {data: "data3",next: node4};
var node2 = {data: "data2",next: node3};
var head = {data: "data1",next: node2};

var current = head;
var length = 0;
var middle = head;

while (current.next != null) {
    length++;
    if (length % 2 == 0) {
        middle = middle.next;
    }
    current = current.next;
}
console.log(middle);



Other programming challenges: 

Thursday, June 18, 2015

Can your tests survive the coming mutant apocalypse? [REPOST]



Are your unit tests effective against code mutations? No, you don't need to worry about your code being bombarded by gamma rays or being bitten by a radioactive spider. The question is will your tests catch minuscule changes in the code base that could have disastrous affects. Mutation testing has been around since 1971 but the majority of people in the software world haven't heard of it. So congratulations you have just learned something that can wow and amaze your peers at the lunch table.



Breaking code is easy

This may be earth shattering news but software engineers are not perfect. We make mistakes but that is why we test. The code written during a 2am coding bender all hyped up on Jolt Cola may be the stuff of legend that creates the next paradigm shifting application but one typo can result in catastrophe. Have you ever spent hours upon hours debugging a critical defect that resulted in something silly like the usage of the wrong operator or worse a missing semicolon?


100% code coverage != effective tests

Despite our best intentions test code is not treated with the same respect as the rest of your code base.  One hundred percent code coverage is a unicorn, really really rare, in complex applications but even that is not enough to ensure your code is rock solid. Even if your unit tests result in a perfect code coverage report, are you positive all your assert statements are effectively verifying the results? Mutant testing is essentially tests for your tests.

And you know what the worst part is? 
I NEVER LEARNED TO READ TEST.

If you are taking the time to read this then there is likely no need to convince you of how vital testing is. However, effective test writing is a skill/art/science but it is one of the many critical skills in the software engineering world that they likely did not teach you in college. Formal training on unit test writing is virtually unheard of. During code reviews the test code is not given the same type of scrutiny as the production code. Yet the tests are just as critical to the long term success of your project.


So what is mutation testing?

Your tests ensure your application works as you expect it to but what ensures that your tests sufficiently protect your application. Mutation testing is effectively making minor changes to your source code and ensuring your tests catch it and none of those mutants survive. A mutant that survive could one day grow up to be a defect.
mutant survival score = number of mutants killed / total number of mutants

The creation of a mutant army

As previously discussed mutants are brought into existence by sightly modifying your source code with something as simple as changing an equals into a not equals. Doing this for every conditional, arithmetic, relational, and assignment operator is just the beginning but you can see how the number of mutations even for a fairly simple piece of code can grow quickly.

    public int additionOrSubtraction(boolean isAddition, int x, int y) {
        if (isAddition) { //mutant: replace isAddition with !isAddition
            return x + y; //mutants: replace + with -,*,/,%
        } else {
            return x - y; //mutants: replace - with +,*,/,%
        }
    }

The code above could result in 50 or more mutants which your unit tests would then be executed against.  If any of those mutants do not cause a test failure it "survives."

Creating a mutant army is not something you want to be doing by hand so look for tools available in your language of choice. Mutation testing is done almost exclusively with unit tests in mind. That is not to say that throwing a few mutants randomly against your integration tests would not be valuable too but throwing a fully mutant army against your functional tests would be absurdly costly.




Are you scared of the mutants yet?

Do not alarmed. These mutants are here to save you. Send in the mutants! Save your tests. Your project could be one fat fingered key stroke away from disaster.


Find out more: Give your tests mutant powers with PIT [Part 1]




Post with my own permission as the original author.  (Source)

Wednesday, June 17, 2015

Start your own blog today!

I am a big fan of blogging.  Personally I find that it helps me organize my thoughts and gain a deeper understanding of topics. Blogging can also be great for your career as well.  When recruiters contact me they often reference my blog and something they read on it. One day it could lead to a job referral or a promotion from one of my followers.

You account for maybe 0.5-1% of my average daily usage of my humble blog.  Think about that. Blogging does not have to be of benefit to you 100% of the time. If even 1 out of 1,000 visitors thinks you could be a great addition to their team or thinks they would like to work with you then your career prospects could skyrocket.  Blogging opens doors. Nothing beats an in house referral.

John Sonmez (Simple Programmer), who you may have heard me metion a few times, has a free blog course available.  The course is done via email which is perfect for learning at your own pace and not fall off the bandwagon a couple days after you decide to start blogging.
http://devcareerboost.com/blog-course/
There are lots of information about why you should blog. I have not interested in adding to the noise, but here is why I am blogging.  Maybe you well see something that tips you over the edge to make the commitment to sign of for the course.

Professional Growth
  • Blogging improves your writing skill.
  • It improves your ability to explain potentially complex concepts.
  • It helps me learn because I try to "teach" some of the new technologies I encounter as a way of organizing my thoughts.
Career Growth

  • networking - It is not news that the more people you know the more opportunities are available to you.  This isn't just for job searching but your current team needs the best possible talent too and networking helps find that talent.  Hiring someone from within your network is much less of a gamble.
  • marketing / branding - This does not have be something dirty to shy away from.  Branding yourself is just a way to consistently express who you are and how you add value. 
  • visibility - Some of the best engineers I have met over the course of my career have also been the most introverted. Blogging is a good way to get people out of there shell without a need to leave your desk. Not only that but you can reach a huge audience vs just the people standing by the water cooler.
  • resume supplement - Even if someone doesn't recognize you by name they can easily go to your blog and get a taste of who you are.  Job interviews often go quick.  In phone screens you get 10 to 15 minutes to express how you are worth bringing in for a face to face bit with a blog you can extend that engagement.  Your potential employer may even know you are worth bringing in for a face to face interview just because they cam see your value via your blog.
Good for your company
WHAT?!? How can blogging be good for the company you work for? Blogging helps get your name out there, right? In all the ways that blogging is good for your personal career growth it is good for your company because you are becoming a better employee.  Not only that since your name is in someway linked to them it reflect positively on them. The people you mentor, network with, or even just see your LinkedIn profile now see your company linked to it. When you are looking for a new position wouldn't you want to go somewhere you know other talented engineers work? Talent attracts talent. Corporate blogs are almost always boring but a personal blog from an employee of the company can give you a nice glimpse at the culture and people in the organization.
Giving Back
The internet has helped me solve countless problems professional and personally but what have I done to contribute back?  Stack Overflow for example helped me out of many sticky situations but up until recently I never answered one question there.  I never even asked my own question.  I was just a Stack Overflow voyeur. Starting this blog is in part a way to help me give back to the community.  If I am working through a problem chances are someone else will be in a similar spot in the future so why not write it down.

I don't know by this point if you are convinced or not if you should start your own blog or not. If you are ready to start blogging go sign up for John's course and if you are not go read more about blogging on John's site and maybe he can convince you.  If you are still not convinced read John's book and get on the soft skills bandwagon.  Seriously, there is too much to gain and nothing to lose.



My blogging mentor: Daniele Rossi

Thursday, June 11, 2015

Give your tests mutant powers with PIT [Part 2]




Mutation testing series:



PIT has the ability to run mutation test with ant, maven, gradle, and more so it can easily be run via the command line and in your favorite build environment. The cost of entry into the mutation testing world is low for Java with PIT.  


Sample Project:

I created a little sample project to try mutation out which are welcome to clone. It is nothing special but I wanted to play with it somewhere other than on an enterprise application. Mutation testing can take while to run on large projects so I would recommend experimenting with the concepts on a smaller scale.
Sample Project on Git Hub

How to add PIT to a Maven project:

Step 1: Update that pom to include the following: (replacing the project base package of course)

<build>
    <plugins>
        <plugin>
            <groupId>org.pitest</groupId>
            <artifactId>pitest-maven</artifactId>
            <version>1.1.5</version>
            <configuration>
                <targetClasses>
                    <param>com.bobbylough.MutationTestingDemo.*</param>
                </targetClasses>
                <targetTests>
                    <param>com.bobbylough.MutationTestingDemo.*</param>
                </targetTests>
            </configuration>
        </plugin>
    </plugins>
</build>


Step 2: Pat yourself on the back because your project is not setup for mutation testing.


How to run PIT mutation tests:

Running mutations tests are super simple from the command line.

> mvn org.pitest:pitest-maven:mutationCoverage

* One thing to note is the command above line assumes it is already compiled locally.

Depending on your project size this can take a while to run. PIT does display a ascii spinner so you know the mutation (slave) haven't stalled so you remain patient. In the sample project I was explicit about where the mutation report should go by but by default they are saved in target/pit-reports/YYYYMMDDHHMI.

There are a lot of command line options but you can see them on the PIT website. One of the more interesting ones to me was the "mutators" option where you can specify which mutators to run against your code.  There are some which are not used by default like "remove conditionals mutator" (REMOVE_CONDITIONALS).

> mvn org.pitest:pitest-maven:mutationCoverage -Dmutators=REMOVE_CONDITIONALS,MATH

Although the most useful initially may be the "mutationUnitSize" option.  It will help to ensure your mutant army doesn't grow too large by specifying the maximum number of mutations created per analysis.

> mvn org.pitest:pitest-maven:mutationCoverage -DmutationUnitSize=2


My experience on a real project:

The small, but real enterprise project I started using this on had about 800 unit tests and ended up generating around 2,500 mutations. The test build which normally takes about 30 seconds now took almost 9 minutes with mutation testing. So cost of doing mutation testing is nothing to sneeze at but it is worth it. I would not recommend mutation testing to be done as part of a continuous integration but rather as part of a nightly build to generate the reports. The project used for this initial test has 98% line and 100% conditional coverage.  (Yes, as previously admitted I am a bit obsessive about unit testing,)  However much to my embarrassment the mutation coverage was only 88% .

The mutation testing results showed the value of mutation testing so clearly. Silly stuff was missed like a mutant survived where a line of code which added an element to a list was removed. (How could we have missed an assert to verify an element was not added to the list!) Also there were mutants that survived where lines where properties were set were removed and negated conditional mutants also survived.  It is the simple stuff, the seemingly trivial stuff, that gets overlooked until you spend hours debugging a defect which ends with you banging your head on your desk saying how could I have missed that.

The project does unit test at a class level with mock objects powered by Mockito. The thing that could have helped kill these mutations more than anything else was Mockito.verifyNoMoreInteractions(someMock, anotherMock). It is best practice to have it in there but it often is forgotten about once you finish verifying the results.


Tuesday, June 9, 2015

Give your tests mutant powers with PIT [Part 1]


Mutation Testing basics:  Can your tests survive the coming mutant apocalypse? (LinkedIn)




Mutation testing opens exciting possibilities. Let's start doing it.  Now, stop waiting and just do it.

PIT seems to be regarded as the tool of choice in the Java software development world.  PIT has a maven plugin which can help make it a part of your daily builds.  The tool also works with ant, gradle, and more.  There is even a Sonar plugin available to integrate the mutation testing reports will all your other coverage reports. After you read this post go to the PIT website (http://pitest.org/) and check out all it has to offer.

The cost of entry with the Maven plugin is low but it is not low enough for me,  There is configuration needed to get your project up and running with PIT and the documentation leaves a little be desired. However there is almost no cost to you to install the PIT Eclipse plugin. The plugin that lets you do all the same mutation magic but in eclipse without needing to update your projects pom file and saves you the trouble from running on the command-line. It allows you to limit the tests that are used for mutations very easily.  You can get this going in less than 10 minutes in the morning and be prepared to show it to people at the lunch table.

There are disadvantages to mutation test.  For example it is time consuming and resource intensive.  Which is all the more reason why you should ease into it and put your toes in the water with the eclipse plugin.  It will help you quickly see the value of it.

The plugin can be added via the eclipse market place by searching for the name "PitClipse".  If you need more information about it check out the marketplace website: https://marketplace.eclipse.org/content/pitclipse


The results can be seen in a tree structure in eclipse vs using the html report generated via the command line.





The html report can also been seen in eclipse but I personally feel the tree view is more usable.



Don't wait! Install the plugin and get started.  Once you see the mutations that survive from the unit tests you have told yourself are good enough you will be convinced that mutation testing is the way to go.




Give your tests mutant powers with PIT [Part 2]

Wednesday, June 3, 2015

Mentoring via Stackoverflow

It was a slow evening for me so I created a stackoverflow account and started answering questions. For years I have found great answers on the site but never contributed back.  However in the process of looking at the questions coming in I realized this is actually very similar to mentoring junior engineers.  I don't get too many opportunities to mentor because these I end on working in top heavy teams.  As I was answer questions I got a very similar feeling as mentoring.  It seems to me like a good way improve your mentoring communication skills and give back to a community which I am sure we all have used.

Here is my profile on stackoverflow if care to cyberstalk me:


profile for Bobby at Stack Overflow, Q&A for professional and enthusiast programmers

Tuesday, June 2, 2015

AngularJS - a journey towards specalization

In a previous post I spoke about a desire to specialize.  For right now I think I am going to work on being a AngularJS front end and Java Spring middle tier guy.  I have been working with Java for a decade so no problem there.  Also being a full stack generalist for years I have done plenty of front end work but never really mastered the craft.  With a new job I am starting in a couple weeks I will have the chance to put my academic knowledge of AngularJS to use and start building a practical knowledge base.  Who knows if I will want to specialize in this in a year but for now this is a good place to start.

Following a recommendation by John Sonmez, the Simple Programmer, I should not only try to specialize in my skill set but also this blog.   So I will try to focus this blog now on three topics
  • AngularJS - writing about my journey from beginner to master
  • Java - general software development and algorithms
  • Unit Testing - this is my true passion in software development that I can never really escape

Wednesday, May 27, 2015

Software Engineer in search of a specialty

Arguments for specialization - http://simpleprogrammer.com/2015/04/23/im-not-sure-i-want-to-be-a-specialist/

T-shaped skill sets - http://www.coelevate.com/essays/customer-acquisition

I have been reading recently about specializing more as a software engineer. At this time I don't have a specialty at all.  For all of my career I have been a Java full stack engineer working mostly on web applications doing basically what is needed to get the job done.  Being able debug and speak to all areas of the application is valuable but having a focus/specialty would be even more so.  It would feel good to be more than a jack of all trades master of none engineer.  The time in my career I felt the most valuable to the organization was when I technical domain expert for a really complex feature of the flagship product.  With the pace that technological options are growing it is seemingly impossible task to truly know enough. The desire to specialize is there but the opportunity to isn't apparent to me at the moment so I need to steps to make it happen.

Step One:  Pick a specialty

This is where I get stuck and have been stuck for a couple years now.  I have areas of interest but nothing that stands out as something I can dig deep on within my current position so it would be purely on the side. So TBD...  still stuck at step one.  I should send out messages in a bottle  or smoke signals or something.


Step Two:  Obsess over learning the specialty


Step Three:  I don't know yet...

Tuesday, May 26, 2015

Programming excerise - Alternating loops

This is a fairly easy challenge.  I could see this as a warm up question in a job interview.  The solution I provided assumes the lists are the same length.  In an interview it would often start this way and then be asked to expand upon the solution to account for the variation in list sizes.

Write a function that combines two lists by alternatingly taking elements. For example: given the two lists [a, b, c] and [1, 2, 3], the function should return [a, 1, b, 2, c, 3].




List<String> result = new ArrayList<>();
List<String> letters = new ArrayList<>(Arrays.asList("a","b","c"));
List<String> numbers = new ArrayList<>(Arrays.asList("1","2","3"));

Iterator<String> letterIterator = letters.iterator();
Iterator<String> numberIterator = numbers.iterator();

while (letterIterator.hasNext() || numberIterator.hasNext()) { 
 if (result.size() % 2 == 0) {
  result.add(letterIterator.next());
 } else {
  result.add(numberIterator.next());
 }
}
System.out.println(Arrays.toString(result.toArray()));


Related Content: