angular js: ng-repeat no longer allowing duplicates

Just ran into an interesting issue with angularjs combined with local storage.

Some facts:

1. angularjs adds a $$hashKey property to objects in collections that are being used with ng-repeat and don’t otherwise have an identifier configured.

2. the $$hashKey is typically generated by incrementing a shared variable. This is a fairly simple way to reliably generate new ids.

In our app, we are using local storage to continually save changes that the user has made on the page . This allows them to always have a working draft that they can discard at any time. If the browser dies or they go away for a while, they never lose their work.  They always have the option to submit or discard available to them.

Now, if you store an object in local storage, you’re typically storing the whole object unchanged. We were, of course, which meant that we were including the $$hashKey property on all of our objects kept in local storage.

So, when you reload the page, we would load the draft data from local storage, including the $$hashKeys.

For items added that didn’t have a $$hashKey set yet, angular would generate one. Only, because it’s an incrementing value, it would be behind some of those already existing in other tracked objects. Collisions! Duplicate IDs!

This totally just happened:

Error: Duplicates in a repeater are not allowed …

Oh NO!

In angular 1.1.4, it now throws an error when there are duplicate objects in an ng-repeat, where this was previously tolerated.

There are a couple of ways to get around this.

In our case, we only ran into this issue because we were storing our objects in local storage and then retrieving them again later on new page loads.
We were storing our objects as JSON, and so we were using JSON.stringify and JSON.parse when we get from local storage.  So it was easy to extend our stringify method to strip out the hash properties by passing in a replacer, like so:

JSON.stringify(value, function (key, val) {
     if (key == '$$hashKey') {
         return undefined;
     }
     return val;
});

When we load objects and feed them to an ng-repeat, angular will add hash keys as it needs, so this works fine for us.

But… there is an easier way if you expect duplicates to be possible in your collections. It’s just hard to find. Sometimes looking at the source really is the best documentation…

Anyway,  one way you can handle this is to tell angular that you’d rather have the ng-repeat treat the items’ indexes as the id rather than expecting an id (the default, which like in our case results in generated hash keys when there was no id).

This can be done by defining the ng-repeat using the following style:

<ul>
    <li ng-repeat="thing in things track by $id($index)">
    .....
    </li>
</ul>

Whammy.

Lesson learned, sometimes you’re not just storing what you think you’re storing. You might also be storing framework state information.  And that state may not always be valid.

What are you learning right now?

I’ve been thinking more and more about the active pursuit of learning and improving one’s self.

I’ve sometimes thought about the idea that there are 2 types of people, active and passive thinkers.

Active thinkers are curious, self-critical, and creative. They are continually re-evaluating themselves and trying to improve.

Passive thinkers rely on external motivation. They coast until some person or external circumstance pushes them to think about something.

Way too many of us are currently stuck on passive.

In the tech industry, change is the norm. New technologies and ideas pop up all the time and we are expected to stay on top of the current trends. Without a constant pursuit of learning, we can’t be expected to recommend the best solutions or technologies, and certainly can’t be trusted to effectively build systems.

Learning in a constantly changing landscape could appear to be an overwhelming task.  After all, if what we learn now is going to change soon, why spend our precious time? If we’re good enough at learning on the job, maybe we can just pick up things as we need them..

If you’re not actively learning, you are holding yourself back. Scratch that. You are holding yourself, your team mates, and your company back. We all learn by challenging each other and helping each other to be more awesome. Great people want to be surrounded by other great people and do awesome things.

I’ve often thought that the best solutions come out of great debates.  When you have a team full of active thinkers that understand their craft and take ownership of a project, when everyone is learning, teaching, and challenging each other, amazing things are possible. Isn’t that the team that you’d want to be on?

Or maybe your career goal is gradually becoming the irrelevant old guy that still thinks and solves problems like it’s 1999.

So, if learning and pushing ourselves is so important, why doesn’t everyone do it?

It’s hard. Harder than watching TV and sitting on the couch. Harder than drinking beer. Harder than just showing up in the morning, leaving at 5, and collecting a cheque. Certainly harder than making excuses. After all, we were born awesome and life is short.

Because learning and practising our craft takes so much time, right? And those of us with other hobbies and family just don’t have enough time, right? Right…

When I hear “I don’t have enough time”, to me it really translates to “I don’t know how to manage my time”.

How much time does the average person spend watching TV, aimlessly surfing, playing games, etc.?

We waste an incredible amount of time every day. Hours. And you don’t even have to give up all of that. You could probably still survive just wasting slightly less time.

I don’t think there is anyone who couldn’t make an effort to put in half an hour a day toward learning and improving themselves, investing in their career, and in those around them.

It’s not about brute force number of hours!  It’s about consistency.  Consistently putting a small amount of time and effort into developing yourself will pay off huge over the long term. Do you really want to be the guy (or gal) who is completely oblivious to industry trends, only understands things at a basic level.. and is pretty sure he’s awesome ?  Or do you want to be the guy (or gal) that actually is awesome, reliable, knowledgeable, and is respected by the awesome people around him (or her) for contributing to the awesomeness that they create?

There are lots of ways to learn:

  • Read – books, blogs, twitter. Try to understand and contemplate the thoughts of others.
  • Start a hobby software project. Create something. Be responsible for it’s evolution.
  • Contribute to an open source project. Learn from others.
  • Code katas – practice solving problems with tools & languages of your choice
  • Learn a new language. Try a different approach to thinking about software.
  • Join a group. Meet people. Learn from them.
  • Give a talk or teach something to others. Explaining something can force you to understand something more fully and think about it from a different angle.
  • Write. Revise your thoughts and opinions and put them out there. Listen and learn from feedback.
  • Think and formulate opinions. Discuss with your peers. Debate. Learn from them. Teach them.
  • Other

If you’re not doing any of those things, YOU ARE DOING IT WRONG.  You are being left behind.

You’re stuck in grade 1 and your peers that are actively thinking and learning are moving on to the next grade. You’re a passive thinker, waiting for someone else to tell you what’s important. You are getting older and no better at what you do.

So get out there. Do something. Make yourself better. Make your peers and your company better. They’ll make you better in turn. And then you’ll do great things together.

Or you could just watch more TV. You choose.

Don’t call me a…

Don’t call me a .NET Developer. I just happen to be working on building an application that is built to run on the .NET Framework.

Don’t call me a C# developer. I just happen to be building an application that utilizes C#.

Don’t call me a JavaScript Developer, though I’m writing JavaScript right now. Wat?

Don’t call me a web developer.  I’ve built desktop applications, background services, tools.

I’ve built applications in Java, C, and C++ and nobody died.

I’ve added new functionality to legacy VB.net applications, implemented domain specific languages, helped a team write a compiler for a subset of Pascal.

I’ve written code in Prolog and Lisp.

I’ve solved a few Euler problems with Ruby. I’ve written a tiny bit of F#.

I’ve maintained build scripts written in Python.

I studied Computing Science and have a love for the essence of software. I’m a developer. My skills are focused around problem solving, adapting and learning, with a background in the foundations of computing.

An implementation in a specific language is just one method of solving a problem. A detail.

Languages are tools. The essence of software development for me has always been about building something, creating something to solve a problem. Taking an idea and forming it into something that we can interact with or does work for us.

I don’t care if I start a new project tomorrow in a language I’ve never used in a foreign environment with tools I’ve never touched.

I’ll figure it out. I’ll make the code do what I want. I’ll learn.

And I’ll love it, because that’s what we do…

New Project. Starting on the Right Track.

So, it’s a new year and I’m working on a new project for a client.  Once again, working alongside Mr.Visser.

This time, we’re tasked with redesigning and rebuilding an existing application for a client – “doing it right” was one of the ways it was explained to us.  Of course, we’re up for it ..

When we initially started the project, one of the first expectations was that we would create a “design document” – a document that would describe in detail how the system should look and behave.  I’m a big believer in the idea that the beginning of a project is the point of most ignorance.  Design documents tend to be a bit too waterfall.. and have a tendency to leave the actual users out of the conversation.

So, we proposed creating a functioning mock up instead.  We would build the front end in a couple of days (no longer than a document would take) and provide a clickable demo to garner feedback and ensure that our understanding was correct and we were headed in the right direction.

We set off using yeomanbootstrap, and angular js to build a demo, along with some mock data to aid in simulating what the actual users could expect. After a few days, we were ready for our first demo.

We demoed first for the person charged with managing the project.  After seeing the demo and giving us a bit of feedback, he brought in a user for another demo the next day.  As we continued to refine, a couple more users were brought in.  We knew we were on the right track when we saw that users who had seen the demo already were smiling and waiting on the expressions of the people who hadn’t yet seen it. But one of the users of the old system quickly noticed that we were missing an important field.

The old system had a list of images that the user could select and re-order.  The main user complaint about the old system was that the list of images was very cumbersome to work with.  To re-order them, the users had to remove all of the images and then re-add them again in the order that they wanted.

We thought we could easily come up with something much more elegant.  Drag and drop!  You could do that?  We sure can. And we would make a conscious effort to keep the interface beautiful and uncluttered.

We had mocked up the simplest thing that we liked :

But as our demo had allowed us to communicate with the users and get almost immediate feedback, we were quickly aware of the limitation. The photos needed captions!  Monkeywrench!  How do we add a text box to this list for every photo without making the interface suck?

Well, it would satisfy the business requirements. But. UGH. That looks like barf.

Designing interfaces is hard. We want people to be happy when they use the stuff we build. It’s something that needs to be iterated on just like the rest of an application.  You need to think and then re-think, and then continue to re-evaluate as you learn and keep coming up with better ideas. Bad interfaces make users unhappy and waste their time.  For a business, time is money, so software that wastes time is incredibly expensive.  We didn’t just want to throw something on to meet the requirements.  We wanted to keep it elegant.

And so we thought about it, tried a bunch of stuff.

Dialogs for editing captions?  That keeps the list clean, but now we have hidden information.  The user actually can’t see all of the important stuff when they look at the page. Bad.

After much thinking and experimentation, we eventually came up with something like this:

          

Finally, something that we were happy with.  It still looked nice and satisfied all of our requirements.  Captions were generally a very small amount of text, so showing a large box was unnecessary.  Using CSS transitions and angular js directives, we could easily make the currently selected box smoothly animate to full-size while the others fade into the background during edit.  An HTML placeholder makes it obvious what the box is for when there is no caption.

So glad we didn’t compromise and tack the box on .

And so, the first couple weeks of this project have got my brain thinking about certain things as well as learning some new tools.

These are the things I’m thinking about right now:

1. Functioning demos are way better than static ones for building excitement and getting the right kind of feedback.  If we hadn’t built that initial interest, the right users might not have been pulled in so early and we wouldn’t have caught the missing element so soon.  We’ve made a point of doing this with our last two projects and both times this approach worked extremely well. Demos have a way of attracting people to give feedback and generating excitement, while detailed documents have a way of boring them while they wait to see something real.

2. Design needs to be iterated on just like the rest of the solution.  You rarely can fully understand all of the technical problems, user frustrations, etc. up front. And your first idea is usually not the best.  Iterating on something gives you freedom.  If you spend a month on a design up front, you’re going to have a hard time convincing the customer that doing something different is a good idea. After a month (or week) of work, they expect it to be right.  Without investing too much in a specific idea, you keep the freedom to throw it away and replace it with something better.  With freedom, you don’t have to compromise.  You don’t have to tack things on to an existing design to meet new requirements.  Continually re-visit the user interface design as requirements change.

3. Yeoman, bootstrap, and angular js are pretty awesome for rapidly iterating and building beautiful client-side web stuff.  Yeoman does a great job of automatically refreshing the page when any file changes.  Using stubbed data, this is fantastic for quickly building a mockup.  Anything that speeds up the edit-debug cycle makes you more efficient and more willing to tinker and continually improve your code.  Angular js helps you write clean JavaScript while avoiding having to write wasteful plumbing.  Bootstrap gives you a great set of styles and components to help you avoid starting from scratch and spending that initial tinkering time to get your application looking professional.  Your choice of tools (and proficiency in them) all have an impact on how quickly you can iterate and impress.

Working On-Site is as Awesome as You Are

I am just in the final stages of wrapping up a project where I worked with a client on-site for several months to deliver the solution.  Our company has a fantastic office, and so sometimes it is difficult to give up the comforts and the routine we are accustomed to for another location.  I’ve been thinking about this a bit and decided to post some ideas on making the most of working on-site.  Here are a few key things that I’ve thought about :

1. Expect a bit of culture shock.  Every company has its own culture and way of doing things.  For better or worse, things will be different.  Expect it.  Embrace it.  Take it as an opportunity to learn more about how things are done in other places.

2. Be ready to adapt.  You are going to have to be able to quickly adapt in order to be productive.  Initially, listen more than you talk.  Ask questions.  Think carefully to ask the right questions.  First try to understand how things are done and why before you go assuming that they are wrong.

3. Work in a way that builds trust.  You might have the respect and trust of your co-workers and colleagues, but you will always have to build it in new environments.  Don’t demand or expect respect and trust – earn it.  Take pride in your work.  Do a good job and people will be more willing to listen to your advice.

4. Get to know some people.  I like to make a point of eating with the people I’m working with whenever possible.  I don’t know of a better way during work hours to get to know people.  If you’re going to spend all day with people, they might as well be your friends.  Sitting at your desk and eating your lunch by yourself is for lamers.

5. Find some things to look forward to.  Maybe it’s the elderly gentleman that sells buns every week for a dollar.  Maybe it’s the dude with a good sense of humour and an eye for detail.  Maybe it’s the drive in and a stop at your favourite coffee shop. Maybe it’s the beetle that lives in the bathroom.  Who knows!  But I bet there are some things that you can think of that just make every day that little bit more enjoyable.

In summary, maybe attitude is everything?  Maybe anywhere can be awesome.  I don’t know.  I haven’t been everywhere.  Just don’t be lame.  If you’re bored, you’re boring.  If you put a bit of effort in mixed with a good attitude, you’ll be sure to find something to enjoy.

On Evolving Solutions

Healthy systems aren’t just imagined up front and created as elegant solutions to a problem —  elegant solutions are the result of evolution.

When you complete a story, a feature, a part of a system, you are doing so with the knowledge and ability you have at that moment in time.

It might fulfill the requirements. It might look great from the outside. But we are the watchmakers that care about the inside of the clock. We care about how concise and maintainable our systems are, how testable they are, and how well they can react to change.

When you initially complete a piece of a system and go on to add value to other components, you will learn. Inevitably, the problem domain will become clearer.  Over time, your techniques mature and you learn new ones. You re-think the work you’ve already done and re-evaluate it against the things you now know.

As you improve, you naturally find ways that you could improve problems that you have already “solved”.

You might think that you’ve solved something elegantly, beautifully. And that might even be true given your current understanding. As far as you know, your code is glorious. But your future self mocks you, feels sorry for you for your ignorance.

So, that’s depressing.  Or hopeful?

As you grow and get better at what you do, you’ll inevitably discover better ways to do things. Your future self is embarrassed by the code your present self writes. And that’s a good thing. That means you are improving. So know that whatever you write now is worse than what you will write in the future, but better than what you were ever capable of before.

And so, when you begin to think about solving a new problem, keep this in mind. You don’t have to focus on getting it perfect up front, or even the first implementation.  Because you can’t. You’re not ready. It’s OK to move on to the next piece knowing that you will learn and improve, and that knowledge will help you to make the current piece better. You know that your future self will be more able to make good decisions about the problems you have right now.

Always focus on solving problems with the minimum complexity you can get away with. Solve the core problem only.  Gold plating is evolution’s enemy. YAGNI. And if you don’t need it or don’t know if you will need it, don’t write it until you do know. The less you write now, the less you’ll have to fix later.

Defer decisions until they are necessary. Prioritize.  Always do the most important things first.

Continually re-evaluate.  Refactoring is evolving.

That doesn’t mean that it’s always worth your time to go back and refactor or re-write pieces you’ve already completed. Sometimes that just isn’t providing value. But, inevitably as new features are added to a system, some of those changes will have an impact on pieces of the system that are already considered “completed”.

Remember that in TDD, the refactor step doesn’t only apply to the code you just added, but also how that code integrates into the system. Don’t be afraid to continually improve other pieces of the system as you add value.  That’s how you keep a system healthy and maintainable under change.

SCNA 2012

I was lucky enough to attend the  SCNA (Software Craftsmanship North America) conference this last weekend in Chicago.  Overall, it was a fantastic experience.

The venue was great.  You really can’t get too much better than the view from the Mid-America club on the 80th floor of the Aon Center:

2012-11-10 16.45.44    2012-11-09 07.52.22

They also did a great job with keeping things short.  Every session was a half hour, with good breaks and interesting lightning talks and games in between.  The format really focused on getting the meat of messages across and limited filler required for sessions in longer time slots. I felt this really helped me not to feel tired (or bored!) and stay engaged and interested throughout both days of the conference.

The conference didn’t focus too much on technical detail, as it’s expected those in attendance are already focused heavily on that aspect. It really focuses on discussions centered around how we can improve ourselves and continue to elevate the standard in our craft.  It was refreshing to attend a conference that wasn’t filled with marketing fluff or heavy technology biases.

I met some good people, had some interesting discussions, learned some new things, and felt rejuvenated and motivated after.   The talks by Uncle Bob Martin and Leon Gersing were highlights for me and gave me a lot to ponder.

All in all, I would highly recommend this conference.

Cleaning the Mess

Recently I’ve had the opportunity to work on a project beginning in a curious state.

An offshore team had failed to deliver and we were brought in by a client to “fix and complete” a rather large code base within a fairly aggressive timeline.

My initial instinct was to treat the code fairly conservatively, to avoid deviating from the existing pattern, get it working, and then re-evaluate, refactor, and clean up the code base.  There was a large amount of copy-paste throughout resulting in a lot of duplicated code. A large portion of the system didn’t work at all and a lot of new code would still have to be added.

My instinct was to treat the system as legacy code, avoid large scale changes until the entire system was well-understood, and apply a “red, green, refactor” type of approach on a larger scale across the system.  We were currently in the “red” state.

I had the opportunity to work with a new team member, @stephenvisser, who was much more interested in approaching the system in a more aggressive manner – do a refactor first, minimize the code base, flush out some reusable components and then make it work.   Initially I was a bit hesitant on this approach given our timeline, as I believed it could cause some up-front re-work of pieces that may have already worked and introduce some risk.

He was quite persistent and we eventually agreed to pursue this direction.  We had a lot of code in need of some love.  A depressing mess.

We started extracting some components, cleaning up existing code, and patterns started to evolve.  Some time passed and we had almost halved the code base, removed duplicate code, and were now progressing much faster with new functionality than could have been possible in the old code base.  It was actually becoming fun to fix old code, often times removing 75% of the mass of a function or class.

I had to concede that although a more conservative approach may have eventually yielded a similar result in the end, we certainly could not have been as efficient – and we would have had to work with a messy, bug-riddled, copy-pasted code base for a slightly longer period of time.

Through working on this project, I started to ponder a few things in more depth:

- Forget ego.  Teams need to be able to discuss, debate, and move forward together on solutions.  Don’t get stuck on your initial opinion just because you’ve done it before.  If you’re willing to try an approach that’s new to you, you just might learn something.

- Sometimes learning isn’t just about adding new techniques, technologies, it can also be about re-evaluating what you think you already know and learning more about when it’s appropriate.  Sometimes there’s more than just knowing a solution to a problem,  we also need to fully understand the pros and cons and when one method might be more advantageous over another in a specific situation.

- Technical debt sucks.  I’ve always been a fan of refactoring after, but refactoring out a mess initially, cleaning house before tackling a problem, can provide immense value. If the code base makes you sad, clean it now.  If you can constantly be proud of what you’re working on, you’ll be more likely to own it and continue working to make it the best you can.

If you’re working in someone else’s mess, it might be easy to find reasons not to clean it… but as professionals, we need to focus on making everything we touch better than it was initially, no matter who created it.  Managers and peers can sometimes underestimate the advantages of clean code, the advantages to maintenance, reducing complexity, and in turn the reduction in the likelihood of bugs.

I for one am enjoying thinking of clean code and our integrity as developers from a slightly more aggressive angle. I always enjoy working with others that bring new perspectives and ideas to the team.  Thanks @stephenvisser for making this project more enjoyable!

Mail Service Refactor Proposal

Everyone who’s every moved knows how unreliable mail can be.

In Canada, if you move, you can tell the post office to redirect your mail for a limited time period so that mail addressed to your old place will be redirected to your new place.  During that time, you are expected to contact everyone who will send you mail and tell them your new address so that after the mail redirect runs out, your mail will continue to find you.  All well and good, but a lot of work… you need to tell the government, get a new license, update insurance, bills, clubs, car service, magazines, etc., etc., etc.  And we know that we never quite seem to catch them all.  When you move into a new place, there is a time period (sometimes years) where you continue to get mail for the previous tenant.

But this is the 21st century.  Why is this so hard?  This system worked OK in the past, but surely we can come up with something better.  We don’t have that problem with email.  Wherever I go, I can get my email.

What if we could improve this?  Wouldn’t it be nice if we could just update our address in one place and everyone who sends mail to us would just get the update?  The truth is, I don’t think that would actually be difficult…

Here’s my proposal:

Imagine that every person just had a number.  Or a unique string of numbers and letters.  When you are old enough to get mail, you sign up online with the postal service, and you are assigned a “Mail ID”.  You are expected to log on to the service and just keep your address up to date.

Every person, company, or entity that can receive mail gets a Mail ID.

If I want to receive mail from someone, I give them my Mail ID.

Let’s say Bob Smith and I are pen pals.  We want to write letters to each other.

I’m going to send the first letter so I ask Bob, “Hey Bob, what’s your Mail ID?”.

He tells me his Mail ID is “123-45678-ABCD”.

That’s all I need to know, and so I start writing my letter.

I put it in an envelope,add the necessary address info (not much), throw a stamp on for good measure, and drop that in the mailbox.

Image

Now, when the post office receives the letter, it scans the Mail Id, looks up the recipients current address and prints it on the envelope below the Mail ID.  Now the envelope looks like this:

Image

Whammy.  Address added to the envelope, postman delivers the same way he always did.

Anything sent to Bob always gets sent to his current address so long as he keeps his address updated.

But the beauty is that there is only one place to update.  Log on to the post office site, update once and you get your mail.  No temporary redirect.  No change of address here, there, everywhere.

You would still have an address.  You can still give that out with invitations to your dinner party.  You just now also have a more reliable (and simpler to update) way to get mail sent to you.

Developer Superstition

Maybe you’ve heard the old “reboot your machine and see if it works”.

I know I have… many times.

Don’t do it. To me this is equivalent to giving up on your problem-solving ability and a potential opportunity to learn something. Basically, “I can’t figure this out. Reboot and maybe it will magically start working.”

Ok, so there’s no real consequence to rebooting your dev machine… but are you going to do this in prod? What if there are other applications on the machine, now all victim to your careless superstition? Computers aren’t magical beings with feelings that affect how they react to input. They’re pretty deterministic, and things tend to be pretty repeatable with a given state and input.

If you wouldn’t do that in prod, then why settle in your dev environment? Take the time to understand the underlying problem. If you understand it, you might be able to prevent it from happening later, in prod, when it matters. Never skip a chance to learn something… and remember that every time you give up and reboot, a very cute and innocent puppy dies.