Using Cloudflare as a CDN – a review

Recently one of our clients was experiencing an increase in site downtime. During our investigation of the outage incidents we discovered that the site was increasingly becoming a victim of DOS (denial of service) attacks.

From the data we looked at it appeared the ‘hacker’ would trawl the site, honing in on pages which had the longest response times and then repeatedly hit those pages with requests using up resources on the site and eventually causing the CPU on the database to max out and the site to go down.

Our client hosts with Rackspace who offer a security solution so we asked them for pricing.  They suggested that their managed service would be rather expensive  for our needs and recommended we take a look at Cloudflare.

Cloudflare offers a low cost (entry level plans are free) Content Delivery Network which enables you to save bandwidth and reduce requests to your server by caching some content. In addition (and this was the feature we were most interested in), Cloudflare offers built in security protection to guard against DOS attacks.

Both the caching and security settings are highly configurable through an easy to use interface, help documentation is clear and well written and support is good (support tickets are prioritised according to the plan you’re on – support for clients on paid plans get priority over those on free plans which seems fair).

Cloudflare is amazingly simple and low risk to implement. The most simple way is to simply delegate the top level domain DNS e.g. example.com to Cloudflare who take over the management of your Zone file. You can then choose which of your zone file entries you want to send through Cloudflare and which you don’t. You can set Cloudflare up ready to go with all services in ‘pause’ mode which means when your DNS does initially point to them they don’t do anything other than relay requests.

If you (or your IT department) aren’t happy to delegate the entire DNS for your domain (maybe you have internal systems running on that domain) then it is possible to get a CNAME record setup by Cloudflare for a sub domain e.g. http://www.example.com. This is the route we needed to go down for our client and this option does require you to be on a paid for plan (we went for Business at $200 per website per month).

The steps we followed for implementing Cloudflare were as follows:

1) Setup Cloudflare account and add card details for paid for plan
2) Requested CNAME record from Cloudflare support (we got this in 24 hours)
3) Given a TXT record from Cloudflare to add to the DNS for our example.com domain to allow them to take control
4) When that was done, Cloudflare gave us a CNAME record for the DNS record
5) Client reduced the TTL on the domain
6) We setup all the configuration of the http://www.example.com domain in Cloudflare but set it to ‘pause’
7) Client added the CNAME record to the DNS and once we’d waiting for the TTL to expire we did a tracert to see that we were actually pointing at Cloudflare
8) We then did the cool bit which was pressing the ‘unpause’ button and sending users through the CDN

We gave the site a smoke test and everything seemed to be working as expected. During the day we then proceed to ‘tune’ Cloudflare by gradually turning on the various options that allow you to cache static content (Cloudflare provide a handy list of file extensions it sees as ‘static’ files and you can use page rules to bypass these or to cache more file types).

Each time we made a change we checked the site and made sure everything looked OK before making the next change. We also checked that real traffic wasn’t being blocked by looking at Google Analytics to ensure there wasn’t a sudden drop in activity and asking Rackspace to ensure that all Cloudflare IP addresses (again there’s a useful list) were whitelisted.

At the end of the first couple of days of using Cloudflare we had enough data to see that it was making a difference. It had saved lots of requests (almost 50% of all requests were coming from Cloudflare cache) and had blocked over 100 threats (with the security setting on ‘low’).

dashboard2

dashboard1

The website ‘felt’ much, much faster from a user perspective although our external monitoring wasn’t reflecting this which somewhat confused us. This must be something Cloudflare get asked about on a regular basis and they give a very clear response to this at http://blog.cloudflare.com/ttfb-time-to-first-byte-considered-meaningles/.

So the site was faster, we were blocking some hacking attempts, we were saving bandwidth all looked good. However we looked at IIS logs and could see that we were still getting some bad http requests (PROFIND, COOK, OPTIONS requests for non-existent URLs) and attempts to do some XSS and SQL injection. Our site/code was rejecting these requests as our IIS filters and security settings meant the hackers weren’t getting anywhere but we ideally didn’t want these requests hitting our server at all and wanted Cloudflare to catch and block them.

We then took advantage of the Cloudflare WAF (Web Application Firewall) and this is now blocking most of the ‘dodgy’ looking requests we’ve seen in our IIS logs. We’ve raised a support ticket with Cloudflare support about the few remaining dodgy requests and they’ve responded very promptly to say they will add a WAF rule to block those. If they come through on that promise we’ll be very happy.

wafrules

All in all, Cloudflare appears to deliver on it’s promises, is incredibly easy to setup and configure and support seems good.  There are lots of options we’ve not explored yet such as using their API to automatically clear the cache on a publish from Sitecore which would enable us to cache more than static content.  For a relatively low cost it certainly seems to offer a good alternative to Akamai.

Going backwards is sometimes the only way to go forwards

I was recently drafted in as Project Manager on a redesign project for one of our clients. The project had been underway for several weeks, the team were all very busy, the ‘work in progress’ boards showed that lots was being worked on but the ‘accepted’ and ‘live’ parts of the board were worryingly empty. There was a project plan but it was clear the team weren’t delivering against it.

I observed what was happening in the team over the course of the next week or so whilst trying to get my head round the requirements. The team were doing daily stand-ups reporting in on what they were working on, everyone seemed to be working on unconnected component level user stories.

The team were complaining about the designs, they were inconsistent, kept changing, looked OK for print but wouldn’t work for the web, didn’t work if the content (which was content managed) got longer etc. etc. etc.

Many stories had been started but couldn’t be finished because they were blocked for one reason or another. If the team got blocked, they just started something else.

The team were demoralised, they weren’t getting any sense of progress, couldn’t see the big picture and nobody was doing anything about their frustrations.

One day I asked a question about how we were going to put the redesigned site live. The customer was going to need to do some content population in the live CMS so we would need to let them do that without the risk of the new pages going live before the client was ready. Some of the content on the existing site was going to be re-used but would need to be available in the new designs. Stakeholders would want to be able to see and approve the new site before it went live but this categorically couldn’t be on a URL that just anyone could access – even within the organisation concerned access to the new site would need to be restricted.

When I asked the question everyone in the room looked blank, the team hadn’t thought about this, nobody had asked the question before and there were no ideas.

So we stopped. Mad as it sounds, on a project which was running behind time, running over budget and with nothing actually delivered we decided to stop work. Anything we did was pointless, we weren’t getting anything accepted or live and we had no plan for getting things live.

We don’t like to deliver bad news, but if we have bad news we don’t try and hide it and we deliver it early. We called the customer, we told them we had a problem, we said we were behind plan and that we were going to stop and re-think the approach. Understandably they were concerned, they were adamant that the dates we had given them weren’t moveable, they said they didn’t want to have another call like that again and that they wanted us to update them when we had a new plan.

The senior developer and I then got together and talked about the problem. We took a fresh look at what we needed to deliver and simply printed out in colour all the pages we needed to build. Whilst our backlog of component level stories looked scarily unachievable, the number of pages we needed to build looked far less daunting.

We had a team of 3 developers on the project. All we needed to do was a page a day between the three developers and we’d still have 2 weeks before the go live date for amends, launch plan and as contingency. Was a page a day feasible or should we ask each developer to produce one page each every 3 days? We talked about the strengths of the team – one of the team was a whizz at HTML/CSS, another had a lot of Sitecore backend experience, another was a good all-rounder.

The Senior Developer bought the other developers into the room and put the ‘page a day’ challenge to them. They agreed they’d all prefer to work together, doing the things they were each best at and they were happy to commit to doing a page a day with a ‘we don’t go home unless we’ve met our goal’ attitude. They were fired up, they had something tangible to aim for, it would be really easy for them to know if they’d achieved what they wanted to.

My job as the Project Manager having got that commitment from the team was to make their jobs as easy as possible – clearing the path ahead of them.

I talked to the client about the inconsistencies in the designs and we consolidated a number of the components which were similar thereby reducing the work that was needed (and producing a better user experience). I asked the designer for an updated style guide confirming which fonts, colours, buttons should be used where and when.  We agreed the style guide was the definitive version of what we should follow even if the Photoshop files we had didn’t.  Before we started work on any page I suggested that we’d do a final sanity check that we were working to a confirmed/signed off design to minimise changes later.

We agreed that any ‘changes’ after delivery would be added to a backlog which we wouldn’t look at until after all the main work was done. We’d then prioritise the changes and decide which were ‘must have’ for launch and get as far down them as we could. If there were more ‘must have’ changes than there was time available the client accepted the fact the date would need to slip but that the slippage would be down to the organisational wants/needs and not our inability to deliver what we’d committed to.

We told the client that we wouldn’t start any story that we weren’t confident we could finish and get approval on. We set the client expectations about the time/effort this would require on their part and got their commitment to make the relevant people available on the right days to sign off the work.

We agreed that getting a page ‘done’ for the customer to sign off was getting it all the way through to a live environment and onto a URL to which we could restrict access. This required a change to the architectural approach and we had to spend several days unpicking and re-doing some work we’d already done. But it was absolutely necessary.

Our new plan and approach massively de-risked the launch of the new site. Code was going to be deployed up to live every day. Those that had access would be able to see the site coming together on the live servers, they could update content on the old site and see it in the new designs, they could add new content. In short on go live day, all we needed to do was a configuration change to point the site URL at the new site we’d created – no downtime for the end users or for the content editors.

With renewed vigour the team jumped on the new plan. We stuck those printed out designs on the wall in ‘to do’, ‘doing’ ‘done’ columns and every day those designs would move across the columns and we’d see the progress. The increased production with the team working together rather than independently was amazing to see. If one person finished their piece of the jigsaw early they’d help someone else out. Every day they were doing a new page, some days they did 1.5 or 2 pages.

Daily stand-ups became unnecessary, the team were working so closely together with the same daily goal that they were communicating non-stop all day across the desks. I didn’t need to ask whether they were on track or how they were doing – I could see and hear it.  The client didn’t need updated project plans or status reports – the teams progress was visible every day by the release of something new.

The team delivered the site on time and under budget. The client was able to use the spare budget to include some changes and new features they wanted. The launch day was as smooth a transition as anticipated, literally a push button exercise to switch the new site over. The client was over the moon. The team were buzzing and had a real sense of ownership and pride in the end result.

It was a real team effort and a project I thoroughly enjoyed working on (even at the most stressful times) with a great team of committed professionals (you know who you are).

Here’s what we/I learnt:
• Spend enough time planning up front – don’t be tempted to dive straight in to the work
• Don’t assume every project or piece of work is the same and that processes which have worked before will work again – look at each challenge with a fresh pair of eyes
• The power of a team is far greater than the sum of the individuals – play to your strengths
• Think about getting things live early and how you’ll launch – de-risk big bang launches
• Don’t start what you can’t finish
• Tackle impediments and issues, head-on (frustrations and moans often hide an impediment)
• If it’s not working don’t be afraid to stop and rip up the rulebook
• Find a way to see your progress – nothing gives a greater sense of achievement
• A problem shared is a problem halved – getting the team ideas and commitment is better than telling them what to do and making the commitment for them

Is it true that those that can do, and those that can’t teach?

George Bernard Shaw famously wrote in his play Man and Superman: “He who can, does; he who cannot, teaches”.

I’m a ‘doer’ – no doubt about it. I’m motivated by getting things ‘done’. Give me a ‘to do’ list or ask me to do something and I’m your man (well woman). I’m not good at delegating (‘if you want something doing well, do it yourself’). I can’t bear to see people stuck for ideas or be indecisive so I’ll jump right on in there with solutions (often too quickly) – every time. Ask me about my plans for the future – I can’t tell you that – don’t you know I have this big to do list which is taking all my time up, I’ll think about the future tomorrow.

However my doing tendencies are completely at odds with the role of Manager I find myself in. Like many managers I’ve ended up in this role because of experience, years of service and the assumption/belief from superiors that because I perform well in a role that I’m qualified to manage others doing that role.

I know I’m not alone – thousands if not millions of managers out there are ‘doers’ – given the choice they will focus on the tasks they need to do and spend little time focusing on the people they’re managing. Perhaps the saying should be ‘Those that can do, can’t teach’.

Teaching (or coaching) is going to require me to learn a lot of new skills. I’m going to need to learn to take time to plan ahead, focus on the future and involve others in my plans. I’m going to need to work on how I provide positive and negative feedback to my team (as opposed to praise or criticism). I’m going to need to fight my instincts and delegate and catch myself before jumping in with solutions or answers. I have to give them space to fail (safely) and be there to pick them up when they do. Most importantly I’m going to need to take the time to take an active interest in my team, listen to what they have to say and find motivation in seeing others get things done well.

Now I know what I need to do, how do I go about it? Maybe if I can create a ‘to do’ list……..

Thanks to Marty Brounstein author of ‘Coaching & Mentoring for Dummies’ for the distinction between do-er managers and coaching managers

The Agile Manifesto – Individuals and interactions over processes and tools

Here at True Clarity we like to think of ourselves as ‘agile’ and we like to keep things simple. When asked ‘What’s your process?’ We have been known to say ‘we don’t have a one, we tailor it to you’.

Whilst our existing customers understand the way we work, for potential customers this approach can cause them to perceive we’re a bit ad-hoc, fly-by-the-seat-of-our pants, make-it-up as-we-go along, cowboy sort of an outfit.

Of course we have a process but it’s so familiar and natural to us that it’s like breathing – we don’t think about it and therefore we’re not very good at articulating it.

So this blog post is the first of what we hope will become a series of posts which talk about our processes and the tools and techniques we use. We won’t be inventing any new processes here – these are things we do or tools we use already, every day – naturally.

“The aspects of things that are most important to us are hidden because of their simplicity and familiarity. ”

Prof. Ludwig Wittgenstein

Project Management: How Much Contingency do I need?

Unless you’re managing a project with zero risk (unlikely) then before you can commit to budget or timelines you’re going to need to do some contingency planning.

How do you know what the right amount of contingency on a project is?  The short answer is you don’t.

If you’ve worked with the client, team and technology before then you can use your previous experience to give you a good idea of costs and the level of contingency you’ll need.

If it’s a new project for a new customer or you’re using new technology or you’re working with a new team or external suppliers/dependencies then the project comes with a higher risk factor and any calculations or knowledge you’ve previously gained are unlikely to apply.

Many PMs start off by adding a blanket contingency e.g. 20% to the costs and time of any project however it is hard to articulate the idea/cost of contingency to a business stakeholder (especially in a pitch) using this approach.

You can make a more calculated estimate as to how much contingency you might need by applying a bit of thinking early on.  Contingency is linked to risk so a good method to use is Risk Analysis.

Some typical risks that you may want to mitigate by adding contingency on a project might include:

  • The scope of the project changing as more becomes known
  • The initial estimates of the work may been inaccurate
  • You’re working with unknown technologies
  • You’re working with third party dependencies e.g. release management, hosting, APIs

There are of course other ways to mitigate risks other than adding cost/contingency to the project which we’ll cover in another post.

A simple way to calculate a contingency would be to multiply the % probability by the cost of impact.  For example a risk probability of 30% multiplied by a cost impact of £10K would result in a contingency of £3K.

You are then able to very clearly show business stakeholders your recommended contingency to mitigate any risk (far better than a blanket percentage).  They are able to make a more informed decision based on their appetite for risk as to whether they want to include any contingency or they may decide to accept the risk and understand that the project may cost more.

Many project managers like to include a second contingency amount, often known as the ‘programme contingency’. This can be used for risks that were not identified at the start of the project and emerge later.  The larger and more complex the project the more likely it will be that you will uncover more risks later on.

There’s no sure fire measure or magic formula to help you establish ‘how much contingency is enough’.  As you get more experienced you’ll be able to make more educated guesses.

The most important thing you can do is to make the contingency visible and get sign-off from your stakeholders (internal or external).

If your contingency is agreed you should keep it as a separate budget from that of your main project and the aim should not be to use it up or count it as profit on a project.  Having contingency budget left at the end of a project will put you in the stakeholder’s good books.

Dealing with conflict – establishing the ground rules

Note: This blog post is a work in progress, it explains where we’ve come from and where we’re currently at and we will update this post in the future with what we’ve learned.

As a management team here at True Clarity we’d all read Patrick Lencioni’s ‘Five Dysfunctions of a Team’ and felt we’d spent a good deal of time building up trust within our team and felt confident we had no fear of conflict.

Our meetings certainly never felt devoid of conflict – there was often passionate debate and raised voices but was it really healthy or were we displaying the classic signs of artificial harmony?

We were seeing signs of ‘passive aggression for example team members not saying what they really felt (either agreement or dissent), ‘forgetting’ to do things the team  they had committed to do, ‘blaming’ of other people or teams for our problems and a general lack of anger or frustration being exhibited.

There were areas that there seemed to be an unwritten rule about which were ‘off topic’ for discussion especially if they related to an area of the business individual team members felt responsibility for or felt were someone elses responsibility.

In order to address this we had a Management Team ‘Mastering Conflict’ meeting last week and decided we needed to just have a frank, open discussion about our understanding of conflict and what we were comfortable with.  After everyone had had their say we drew up a set of ‘Conflict Rules’.

Management Team Conflict Rules

We hold each other accountable for:

  1. The basics (being on time for meetings, no mobile phones or laptops)
  2. Staying results (outcome) focussed
    • Not rambling
    • Keeping on topic etc.
  3. Be Kind
    • No shouting
    • No pulling faces or other non-verbal stuff
    • Don’t talk over people
    • No bitching (observations only)
    • Refrain from banter in meetings (meetings should be professional and formal environments)
  4. No hats (explicitly only) (This means that whilst we may have different roles e.g. company directors, heads of departments that when in these meetings we act for the needs of the many/group goals not our individual goals)
  5. Getting Naked (See Patrick Lencioni)
  6. Everyone gets to talk and listen

Time will tell whether we are able to hold each other accountable to these rules and call each other out if we break them.  Early indicators are that this is not easy and definitely something we need to practice!

Update 10th May – Anecdotes & Tolerance

So how are we going following our conflict rules.  Well we’re all learning to communicate better with each other.

We have found that sometimes people are unable to express themselves clearly in a meeting or are unwilling to pitch in, particularly if things are getting heated.  We have found that writing anecdotes in the third person and making those visible have made it easier for people to say how they feel or what they think in a more considered way which is a good way to open up a dialogue.  We hope that in time these won’t be necessary and we’ll have evolved as a team to an extent where we can say anything at any time.

We have recognised a need for tolerance in the team.  It’s important for us to catch an emotional response in ones self as a sign the other person is trying to communicate better. We need to focus on what is trying to be said (with best intentions) and not try and assume someone is saying one thing and meaning another.

We need to be able to notice fear in the team.  Fear of people disengaging if someone says the wrong thing. or fear of the angry “shout down” if they say the wrong thing.  However recognise we all need to be able to “say the wrong thing” so we can learn to communicate better.  Not saying what you feel/think won’t help the team learn.  We hold ourselves accountable for being honest.

Start with the Why, How great leaders inspire action by Simon Senek

People don’t buy what you do, they buy WHY you do it.  So it’s important to know why as a company you do what you do, then hire people who believe what you believe and find (or convert) customers to believe what you believe.

Great TED talk from Simon Senek with some good examples of why some companies like Apple succeed and others fail when fundamentally they are all capable of producing the same end product/result.

So, what’s the ‘Why’ at True Clarity?

It’s the shared sense of achievement we get from helping the people we work with successfully realise their goals.

Agile Product Ownership In a Nutshell by Henrik Kniberg

A fantastic high level summary of how work is defined, prioritised and delivered in an agile environment to deliver value to stakeholders.  This is only a 15 minute video but the content and visual examples given are excellent.  A must for anyone new to agile or for those wanting a refresher.

Things this video will help you understand:

  • What a backlog is and how to manage it
  • Underststanding the difference between story size and value and using this to prioritise
  • Learning to say no (and why)
  • The importance of communication
  • Being able to forcast how long things may take to deliver based on previous experience
  • The importance of frequent delivery of value for the feedback loop and prioritisation of the next most important thing
  • Managing risk in agile
  • The customer value curve and the fact that knowledge is value

I can’t use CAPTCHA – does that mean I’m not a human?

Am I alone in absolutely hating any form that makes me fill out CAPTCHA?  I consider myself an intelligent, literate person but struggle to read the strange words and numbers that CAPTCHA presents me with.  I wonder how many people fail to complete forms simply because of these horrible barriers?

One of our teams was pondering this problem the other week and came across this nifty alternative for people to prove they are in fact humans.  http://www.areyouahuman.com/.

I absolutely love this and think it’s a much better alternative and likely to make people fill in forms lots just to play another game – they quote a 40% increase in completions – a marketing dream!!

We’re going to be offering this alternative to CAPTCHA to our clients in the future so if this is something you’d like to see on your website, please let us know.  Works on mobiles and with flash or HTML5 and claims good reliability and scalability so looks like a safe option.

One last thing – do read the instructions on the games carefully – I initially struggled trying to put non-food objects in the fridge and to pin flying objects to the ground.  Clearly I’m not as intelligent as I’d like to think I am!