Octopus 3.1 Pre-release variable bug

I recently upgraded to the Octopus 3.1 pre-release to make use of the Azure Cloud Service deployment changes and leverage unpackaging to handle config transforms and variable replacement (which is pretty great).

But then one of my package deployments started running into an issue.

I started seeing an error like this:

The string starting: 08:19:13Error
At C:\Octopus\Work\20150909141912-277\Bootstrap.ps1:17 char:253
08:19:13Error + & "${env:TentacleHome}\Calamari\3.1.0.487-beta\Calamari.exe" deploy-package -
08:19:13Error package "C:\Octopus\Files\***.***.***.1.0.0.764.nupkg-78a7964
08:19:13Error 7-898a-4f72-a826-0705f7a3b3e4" -variables "Variables.json" -sensitiveVariables=
08:19:13Error "Variables.secret <<<< " $variablePassword
08:19:13Error is missing the terminator: ".
08:19:13Error At C:\Octopus\Work\20150909141912-277\Bootstrap.ps1:23 char:1
08:19:13Error +  <<<< 
08:19:13Error   + CategoryInfo          : ParserError: ( $variablePassw...tExitCode 
08:19:13Error }
08:19:13Error :String) [], ParentContainsErrorRecordException
08:19:13Error     + FullyQualifiedErrorId : TerminatorExpectedAtEndOfString
08:19:13Error
08:19:14Error The remote script failed with exit code 1

After some trial and error and some googling and some sheer luck, I came across this issue: Sensitive variables with special character are not working in 3.1.

Seems that is the problem.

One of the variables in my project is marked as sensitive and has a special character in it.

I set the variable back to not marked as sensitive which is OK in my current environment as a temporary workaround, but may not be for others.

Alternatively, you could change the variable value to one without any special characters. But that could have ripple effects if you’re using variable sets and may need to redeploy affected projects.

I expect it will be fixed soon, but in the meantime this workaround should get you back to deploying.

Software is Meant to Evolve

Software is meant to evolve.

As a consultant, I see many organizations that don’t fully consider this when planning and building systems. This has a negative effect on the quality of a solution and can even hold us back, leaving process stagnant and the business vulnerable for competition to disrupt.

We tend to think primarily in terms of projects.

We often see the development of a system as something that should have a beginning and an end. We create a timeline for construction — a short window of opportunity to ensure that everything we could possibly need makes it into the system.

We’re going to cram all of the features we can think of into a project.

Now, I’m not talking about Agile vs. Waterfall. Although agile favours prioritization and evolution, most software initially built in an “agile” way, will remain stagnant for long periods of time after successfully delivering something usable.

The system is expected to “just work” for many years, maybe with a little bit of maintenance. The project is an inconvenient period of construction that, once complete, gets out of our way as we head back to work with our new tools.

After several years, we might decide to repeat the process. We’ll have a big wish list of all the things that users have been complaining about. We’ll take that and start another project to re-implement the system with our new ideas and maybe even some new technology.

Is software like a hammer or a saw that we should use until it wears out? Once it is no longer working for us, we’ll just get a better one.

But software is special.

Software, unlike our physical tools, has a unique characteristic — it can evolve. It’s malleable. No tool in history has had this degree of flexibility.

The fact that it can be modified and improved is one of its biggest strengths.  It can change with us to meet demand and can learn new ways of making us better at what we do. It can evolve and change the way we work, allowing us to focus more on our most important tasks.

It’s also easier than ever to deploy our changes more frequently. With modern software platforms and tools supporting automated deployment and continuous delivery, our systems can be updated even as they are being used!

This flexibility for change, combined with an ability to easily and reliably deploy is incredibly powerful.

Modern businesses are heavily dependent on software. But I have yet to see a perfect system. There are often existing inefficiencies that are affecting employees, costing money, time, or even compromising agility. Being dependent on these flawed systems handicaps businesses and puts them at risk of being left behind or disrupted by those with better tools. We often tolerate these flawed systems and hope to someday build a new system will to replace it.

Failing to evolve is missing opportunity.

Most products that we depend on, SaaS applications for example, improve frequently for their customers. But many organizations aren’t regularly improving internal custom software that is absolutely core to the success of the business.

Technology is intended to give competitive advantages, multiply the effectiveness of our employees, reduce risk of mistakes, give additional insight into decision-making, etc.  If I told you that I knew a secret that could make your employees 5% more effective at their jobs, would you want to know what it is? You’d probably click that link.

That system that everyone is complaining about might be costing us thousands of hours in lost productivity, cleverly disguised as a few minutes a day, spread across thousands of employees. Many opportunities for improvement already exist in some of the stagnant software that you are depending on today. If we can identify them and their costs, we’d be crazy to tolerate them. And yet we often do.

I like to think of a solution as an employee. It could be your best employee. One that works tirelessly day and night, makes everyone who comes into contact with it more effective, and has the ability to continually learn new tricks. A system that you don’t invest in and evolve is more like an employee that is actively disengaged, preventing your other employees from reaching their true potential. Maybe even discouraging them.

If you’re like most businesses, you’re not sending your system to Mars. It’s not going away for long periods of time, untouchable and completely dependent on the decisions we made before it left Earth. Our systems are always within reach and available for change.

We should be thinking about evolution from the start.

We need to recognize that it’s impossible to see the future and build a perfect system up-front that will work gracefully to handle all possible future growth and change to our businesses. Our systems are always missing out on some type of improvement, with the gaps growing over time. They can only remain effective and relevant if they are effectively evolved.

I see creating a system as the beginning spark. It should be thought of as just a starting point for a long-term contributor toward an ever-improving business. If we think in these terms and recognize that our system must be adapting and evolving alongside us, we can recognize that any initial project is really all about properly setting the stage for future enhancements.

When we think about creating something that we expect to evolve, we have different priorities.

We can free ourselves from the pressure of trying to get everything right in a single tragic attempt.

Our projects should become much more centred around prioritization for delivering value to the business. An initial project can focus on a minimum viable product. Subsequent projects or iterations are about continuing to evolve to provide features that maximize value.

Understanding the types of changes we should be making requires constant feedback from users and metrics to determine where real value lies.

For a system to remain relevant long-term, we must also place a higher importance on quality. It is extremely important that our system is able to react quickly to change. Messes that make change difficult can not be tolerated. Our code should be constantly kept healthy.

Our systems should be well-tested and deployment should be so reliable and frequent as to be seen as a non-event. We should be seeing value from change so often that the idea of not updating would be unthinkable.

We need to break the cycles of falling behind and then catching up.

We need to think about software more as an ever-evolving, ever-improving tool and less as a tool that needs updates every few years.

By planning for evolution, we can break these cycles of building giant monolithic systems, using them until we hate them, and then when our inefficiency becomes intolerable, initiating another project to “catch-up” and modernize.

There is a trend in modern software toward continuous updates. Google has all but done away with version numbers with Chrome and has done a great job of continuously updating in the background. Microsoft is working on ensuring that Windows 10 will be updated monthly with new features and enhancements. Forward-thinking enterprises also need to start thinking in this way in order to stay competitive. The businesses that are able to evolve fastest will dominate.

Our systems should be thought of from the start as companions that will evolve alongside us. If we continue to invest in making them better, they will frequently provide us new value and hopefully help us to avoid ever having to play catch-up to our competitors.

Skills Aren’t Binary. You’re Wasting Precious Time.

Skills aren’t binary. You don’t just learn how to do something. You aren’t either an expert or completely ignorant. There are many levels to most skills. And so when you start learning, you begin a journey in developing yourself.

It takes a lot of time to get good at something. Any fundamental skill is something you just can’t afford putting off the development of.  If you aren’t jumping in to learn something that you should be, you aren’t just a little bit behind… you might actually be behind your peers by all of the time it takes to gain adequate experience, which might be several projects, maybe even years. Those that have already started the journey of actively learning are off and running and those that put it off are still standing still at the starting line.

Take a fundamental like unit testing as an example. There aren’t many software development skills that are almost universally agreed upon as essential and beneficial, and yet there still exist leagues of developers that continue to procrastinate and put off the investment into learning and practice.

Learning the skill of testing is a progression through understanding some absolute basics of setting up a project properly, learning and rethinking structure and code to make it more testable and maintainable, different strategies for isolating components for testing, learning how to structure and maintain the tests themselves, becoming effective with new workflows and methodologies, maybe some automation tricks, and a lot of time learning how to distinguish between tests that provide value and those that don’t.

Testing isn’t something you just pick up and do, it’s something that you begin learning that requires an investment in time to get good at. It also fundamentally changes the way you think about and design software and is essential to maturing as a developer and learning how to create quality work.

For fundamentals, there’s absolutely no excuse to put off learning if we ever want to be amazing.

If you haven’t started the journey, you’re probably just falling further and further behind… whatever are you waiting for?

Is an Overemphasis on Tasks a Barrier to Quality?

We love to plan when we build software. We create requirements and mock-ups. We create requirement documents, use cases and user stories. We create backlogs of all the things and we might even break them into fine-grained tasks with estimates.

We need to stay organized right?

We need to track progress and ensure that we know what to work on next. If it’s not in the backlog, it’s not important.

Of course we need to stay organized. And accountable. And we need to show progress.

But maybe our list of tasks or features misses out on one of the most important aspects of awesome software.

In my experience, the best work has been a healthy mix of working on focused tasks and some time spent just improving pieces of the system as I come across them or have new ideas. Improving past work. Sometimes the most important work in a project is the work that we don’t talk about.

What does it mean to “finish” a feature? With an overemphasis on tasks and tracking time against them, are we unintentionally creating a barrier to evolving our code bases? We may not feel like we have the freedom or time to work on anything not immediately related to a task. And we may feel like we shouldn’t go back and work on improving a feature that was “finished” a couple weeks ago.

One of the phrases that stuck with me from one of my earlier jobs was a colleague saying “done is not done” during a stand-up when another colleague said he was done working on a task. He was trying to emphasize that although the feature worked as intended, there was lots of room for improvement. Was it properly tested? Was the code clean? Was it consistent with the rest of the system?

We’re often not accounting for the over-arching continuous improvement task. Should a percentage of our time be allotted for general improvements?

“Always leave the campground cleaner than you found it.”  – The Boy Scout Rule

We come across a lot of related or unrelated pieces of a system as we’re working on creating or modifying features. We might be inspired and discover a better way to do something that we had considered done last week.

We almost never do things 100% awesome the first time. Great software evolves. But when is it appropriate to improve it?

Should we create tasks for every little improvement that we can make and then include them in the backlog for prioritization? We’re just trying to stay accountable, but let’s be honest, tasks like improving readability, refactoring, simplifying, and reducing duplication will likely be at the bottom of the list in a typical team even though they may be relatively small tasks individually. We’ll wait until our technical debt becomes a bit more intimidating.

The expectation of crunching through tickets can create a “not my problem” or a “we don’t have time” attitude within teams and the quality of the system suffers. Software that isn’t continuously refactored and simplified just keeps getting more complex.

So tackling issues sooner often has a payoff. We’re evolving our solution, keeping our code base clean, and culling technical debt early. We’re making future work easier.

Now if we maintain a responsibility to the appearance of only working on tasks from our task list and we also decide to take responsibility for continuous improvement, we’re going to be borrowing time from our other tasks. And so most of our tasks are lies. They’re slightly inflated by the good work we did elsewhere.

That’s great if we can beat our estimates for the things we promised to work on – we might be able to give ourselves a bit of room to keep the system from getting out of control.

But what if we don’t have room in the estimate? What if we’re dealing with a list of tasks with estimates that we just can’t beat and someone’s breathing down your neck. Maybe they were created by someone else. We might be working with new tools or a new team and we’re just barely getting by thrashing through tickets. We might feel pressured to stay late to make improvements and not tell anyone. But our dedication to quality is just helping to propagate the lie that it is possible to do things perfectly the first time.

But we’re scrambling and falling behind. And the quality of the system is spiraling out of control. But at least we’re getting the features completed.

Where does that leave us after a few months? A few years?

Everyone is still expecting quality work from us. We’re just pretending to get it right the first time. Hoping no one notices. Or maybe we naively believe that we will magically be granted time for improvement later. We finished all the features, now we can relax and refactor. Has that ever happened?

Is it possible to be more up-front about quality expectations? How can we emphasize code quality throughout a project? Maybe we need to formalize it.

Maybe we need something like the Google 20% time for code bases. 80% of our time is tracked against features on our backlog. Those absolutely need to get done. They’re the core of what we’re building. But we also have the freedom to spend up to 20% of our time just improving past work and evolving the solution forward. Or 10%. Whatever works.

Maybe “Improving the code base” could even be a formal task allocated each sprint per developer. We’re communicating that this is a priority and we have a bucket to draw hours from so we’re not inflating time tracked against our other tasks. The team has a clear understanding that quality is always an expectation.

I’m not sure, but I think that there is a need to be more transparent and up-front with expectations of quality and account for it more often in software projects. I think that the freedom to improve things also gives a sense of ownership and more opportunity to take pride in our work.

We should be spending (and encouraging each other to spend) a portion of our time just making things more awesome. Continuously.

Godspeed, fellow developers. Godspeed.

Saving the World with Software

There’s been an increased focus on user experience in the last few years, bringing with it an avalanche of related buzzwords.

Everyone’s wants them some UX. No, no… interaction design. Usability?

Anyway, it’s refreshing to see design and the creation of elegant interfaces take greater prominence in building systems. We now judge our software by a higher standard than we did 10 years ago.

We’re now much more aware of how our users are using our systems. And we try to build things that make their lives easier. After all, if we make the end users happy, we’ve succeeded right? Everyone will love us.

Or maybe..

End users aren’t the only people that “experience” our software. If you’re building software for others, we might have to include a few more.. how about everyone that comes in contact with it in some way or another? And then also include the people that come into contact with us during the process.

The operations team might have to run our installation and upgrade scripts, or make configuration changes to the system. They will have some sort of experience.

Future maintaining developers will “experience” all of the ease or pain of making changes, fixing issues, and keeping the thing running. If it’s not a pleasant experience, they’ll make sure everyone knows. All your hard work, your beautiful user experience, compared to lipstick on a pig – a facade that hides some shoddy workmanship. But you worked hard on it, right? You just didn’t have time to refactor that part of the system. But now it’s too late to explain everything to the new developer who just wants to rebuild the whole thing in LOLCODE anyway because it’s clearly a mess now. And he figures it would only take him a couple weeks to do it “properly”.

The project sponsor, overseeing manager, or CEO may never experience using our software directly. His or her experience may be the effect on the business. The bottom line. Does the thing do what it was supposed to? If it doesn’t, you guessed it — bad experience.

What’s the turnaround time on getting something fixed and/or deployed to production? Automating stuff? Yes, we can. If you’re slow or can’t do things reliably and in a repeatable way.. bad experience!

Are you introducing new bugs every time you fix an old one? Oh no! Bad experience all around. We have testing for that.

For some, their experience may be entirely secondhand. Did we increase or decrease the number of people swearing and flipping tables in the office around them? Or are they overhearing users saying that the new system is so much better than what they had before. That might be the only experience some have as they are indirectly affected by the quality of the work we do. You affected the mood of the room. Congratulations!

There are loads of people that we affect daily throughout the processes of designing, delivering, and maintaining software.

Lucky for us, the bar is set pretty low with a lot of enterprise software. We don’t have to be perfect. There’s no such thing as perfect. And we have real constraints like time, money, and resources.

But the beautiful thing about software is that it is able to evolve. We can keep refining it and making it better.  And so we evolve as well. You are continually getting better at your craft, right?

Every time we deliver something, we’re learning from our past and delivering better than ever before.

I’m thinking one of the ways that we can do that is to consciously consider the experiences of all the people that our software affects:

  • End users and everyone within earshot. UX, baby.
  • Administrators, operations. Did they say “wow, that was easy!”. They should have.
  • Management. Or, everyone who sees the progress or status reports. Wow, this is what it’s like getting a professional to build stuff for us.
  • Developers. Maintainers. Wow, your code is so easy to understand, fix, and modify. Respect!
  • Stakeholders. Your work did what it was supposed to. You delivered! Trust!

Are we making their day better, even marginally, by building software for them? Are they better off with us or without?

We’re practically saving the world with software. Except we’re not at all.

But hopefully we’re making it just a little bit better for everyone that comes into contact with our work and us during the process.

Your IT Department is not a Cost Centre

From Wikipedia, “A cost centre or cost center is a division within a business which is financed from the profit margin adding to the cost of the organization, but contributing to its profit indirectly.

Well, OK, technically it might be a cost centre.

But describing something in terms of cost sounds pretty negative.  And I’d even say that if you’re thinking about technology only in terms of cost, or as if it were some mundane necessity to doing business, you are probably doing it wrong. It may be that you have a hard time quantifying the effect of technology on your organization. So let’s go back to the basics. Why do we even need technology in our businesses in the first place?

““Man is a toolmaker, has the ability to make a tool to amplify the amount of inherent ability that he has…what we are doing is building tools that amplify a human ability…the industrial revolution was an amplification of a human ability, sweat…What we are working towards now is the ability to amplify another human ability and we are just starting to get glimmerings of where it is going to go.”” – Steve Jobs

Whaaaaaaat!!!? Think about that.

Every piece of technology that you have should be amplifying the ability of your employees.

They’ve been granted superpowers that allow them to:

  • communicate across long distances
  • work with multiple customers at the same time
  • remember things forever and easily recall them
  • find anything in a matter of seconds
  • fill out reports and enter data at superhuman speeds
  • easily spot inconsistencies and errors
  • be automatically alerted to problems as if they had a sixth-sense
  • easily accomplish difficult tasks directly related to your business

Your website even keeps on working after everyone’s gone home. I bet you don’t get many calls for your address anymore. It might even take orders and put money in your bank account for you. That’s a smart robot.

Thinking in terms of cost is really missing the point.

Everything you do in regards to technology should really be making every other thing more efficient or capable, and therefore, more profitable. If not, why are you doing it?

Cost centre? More like PROFIT centre steroids!

If you had an employee that could enhance everyone around him or her, you’d probably invest in that guy there. You’d probably pay a fine salary if you had someone that could make everyone else better at their job.

But you already have that. You do. You just didn’t realize it. Or you should… you could hire some smart people to help you with that. Think of the potential!

It might need some training. It might need a little investment, a little love. It might need a department-sized defibrillator to bring the life back.

But just imagine what you could do once you get that mechanical heart pumping those digital performance-enhancing substances through your systems…

Customer-First Development

Some things I’ve come across the last few weeks got me thinking.

First, an excellent blog post by Gojko Adzic, Writing As a User” does not make it a user story”

This post reminds us that too often our user stories are lies. We write stories that misrepresent what the user really wants to do and our stories are biased toward our implementation preferences right from the start. We’re polluting our solution before it even has a chance.

We should be brutally honest about what the user really wants to do and then focus on that first. User first. 

Second,  I recently re-watched this old video of Steve Jobs while I was killing some time:

He responds to a tough, condescending question with an epic response.  Most of what he said was about what Apple was trying to do at the time, but there was something he said that hit me because it’s describes something that we’re constantly failing to do well in our industry:

“One of the things that I’ve always found is that … you’ve got to start with the customer experience and work backwards to the technology. You can’t start with the technology and try to figure out where you’re [going to] try to sell it.  And I’ve made this mistake probably more than anyone else in this room and I’ve got the scar tissue to prove it.  And I know that it’s the case.  And as we have tried to come up with a strategy and a vision for Apple, it started with “what incredible benefits can we give to the customer?”. “Where can we take the customer?”, not starting with, “let’s sit down with the engineers and figure out what awesome technology we have and how we’re going to market that”. And I think that’s the right path to take.”

Oh snap. Customer experience first. In 1997.

But it’s 2013, so why are we still delivering things that people don’t like to use?

You mean this thing that I spent the last year building isn’t exactly what you wanted?…

Projects are still water-failing, but we’re supposed to be past that aren’t we? But communication is hard. Responsibility is hard. We’d rather not think about that for a while and go play with our favourite language. We’re pretty clever… we’ll build the most awesomest thing ever. We’ll use TDD, we’ll have some sweet RESTful services and the cleanest API you’ve ever seen.  And we’ll even continuously deliver!

Not that those things are bad. They’re pretty awesome. But let’s at least make sure we’re building the right thing before we continuously deliver something no one really wants.

We still seem to place too much importance on our favourite frameworks, our favourite tools, our development methodologies. We easily get bound to a stack or jump into dependencies on third-party tools before we’ve even considered whether or not we are building the right thing.

Which reminds me of something else.

I really hate seeing projects become too dependent on third-party tools to the point that it becomes difficult to just say yes we can build that. Oh, there’s not a component for that… sorry, we can’t build that. Oh.. I guess we’ll have to tell the users that it’s not possible.

Are we afraid to invent, create our tools, make hard choices, and solve real problems? Are we afraid that when we try to solve real problems, we might need to get dirty?

I do think that there is generally a lack of developers that have the ability and mindset to build whatever is necessary to create the best possible solution to a genuine need.

We have to break out of the mindset that our tools and languages are limiting what we can achieve. We’re the ones limiting what we can achieve when we say things like “sorry, that’s not possible in [some framework]”. If your tools limit you in any way, maybe it’s time to switch.

After all, you can build anything can’t you?

And if cranking out something yourself is truly the best way to solve the problem, maybe you should.

I don’t know. There is a concerning lack of customer-first development in many software projects.

I do know that we should at least be staying focused on building the right thing all the time, continually keeping our goals aligned with building what the user really wants.

Our goal is never to “build a website in [insert technology here]” or “create a mobile app”, but should be to solve a real problem.

So go out there and make users happy.

Question everything. Even requirements! Especially requirements.

Communicate more. Demo more.

Build cool stuff.

Throw out stuff that sucks.

Get real feedback from real users.

Change lives and save babies. And puppies.

Iterate towards perfection.

It’s worth a shot.