BigData, Big CIO Opportunity?

There's enough hype, marketing, and certainly media attention on BigData to catch a CIO's attention. But beyond the hype, there is something very real and very important for CIOs about BigData - a potential opportunity and a possible threat.

I really like O'Reilly's simplified definition of BigData and Introduction to the BigData Landscape. BigData is volume (how much data), velocity (how much new data, and how much does the data change), and variety (both structured and unstructured data). BigData is "hot" because the technologies to solve BigData challenges are more accessible - cloud, in memory databases, easy to use visualization tools, new database options (nosql, xml, ...), and choice between open source and commercial tools. BigData is also Big Business, and is sized to $50 billion by 2017 with both big and small vendors competing. Also, new data and analytical capabilities has the potential to transform entire industries.

BigData also presents both talent and organizational challenges, but more on that in another post.

In talking about BigData with a colleague, I realized what should be important to the CIO today  is that BigData is relative. How much volume, velocity, or variety that "defines" BigData is relative to the CIO's capabilities (both technical and organizational) versus the competition in the industry. So an organization that is lagging in BigData and analytical capabilities is going to find that their competition is smarter, faster, and possibly more profitable. A CIO that can drive the organization's BigData capabilities has the potential to create a strategic advantage versus competition.

BigData is about scale - can the CIO outpace the organization's vision on what it wants to do with data using a combination of talent, technology and process? If the organization doesn't have a "data vision", then the CIO needs to paint the canvas of possibilities by demonstrating new analytical capabilities.

So yes, there's plenty of media and hype, but the CIO has to pay attention.

Top Five Itineraries for the CIO's Offshore Visit

Visiting the Victoria Memorial in Kolkata

What should a CIO say to the troops - the architects, developers, testers, project managers, business analysts, product owners, engineers, support staff - when you have limited opportunities to meet them face to face? I had several of those moments over the last couple of weeks as I visited several development and operational teams that work offshore. I visit these teams yearly, and last year I posted my Top Ten Reasons to Visit your Offshore Teams. With such limited to visit each team, I needed a structured approach and itinerary to these visits. Here's what it looked like:

• Craft the message - I'm usually not a big fan of presentations, but with limited time, this was the best approach to show case several important tools my team had developed. The presentation covered our accomplishments in 2011, our goals for 2012 and our team structure. I then had a few open dialogues around two keep aspects of our technology culture and process - agile development and our software development standards.
• 15 minutes in the life of - I'm an "in the weeds" CIO. I like looking at computer screens and seeing how people do their work. You can learn a lot by just inspecting the code your developer is working on, the test cases your tester is automating, or the tools your support team is leveraging. Believe me, if the CIO can't listen, read, and understand what he or she is seeing, it's even harder for others to collaborate. Also, I like to get a sense of whether individuals are working on the right things, at the right level of quality, with the right tools.
• Meet individuals - Create opportunities to have one on one conversations. Sometimes, you have to make sure the agenda includes these conversations, other times you have to fabricate these moments. The CIO has to help make connections between onshore employees and offshore consultants, so there's no better way than to identify and meet with key individuals while visiting teams.
• Teach the business - The CIO must represent the Business on these trips and recognize that consultants probably have little understanding of the industry, customer needs, competing products, etc. There isn't an easy way to give consultants a full course during a short visit, so I like to leave them with simple "pictures" or "stories" that help them identify with industry dynamics or with key customer segments.
• Learn new capabilities - New capabilities can come from your partners and their vast R&D resources, but it can also come from your teams and individuals. A strong offshore visit should include sessions where new capabilities are explored, a dialogue develops, and ideally new ideas emerge. This is one of the reason I like to bring Business colleagues on these trips.

The other piece of advice I have is to keep a journal of follow ups and action items. I'm still organizing my final list from a trip that I completed last week!

CIO Advice: Listen, Answer The Question, Provide Insights

I was relatively young when I attended my first board meeting. At the time, the board for my startup consisted of executives from some of the major media and newspaper companies including Hearst, Advance, Scripps, and Media News Group. This was back is 1998 or 1999, and like all Boards, they wanted to know how we were growing our business. They had few technology questions, though I do remember one board member asking me to explain browser cookies. So I was mostly quiet and listened during the session.

Now we had two GMs at this meeting, both of them good at what they did, and both were asked to present their businesses. Now if you've been in one of these situations, you know how quickly the Board will send you off presentation. Surely, by slide #2, the questions pour in and the fifteen minutes scheduled turns into a Q&A session. That's usually ok, as long as you know how to manage the situation. You have to answer the questions but still attempt to get your key message out.

Let's just say, one GM was more successful than the other. The advice the successful GM gave me afterward is something that I've remembered since then:

When faced with questions from senior executives, keep three things in mind:

1. Pause, listen, and understand - Most people respond to questions too quickly. The mouth moves faster than the brain, and it is likely a quick response comes across as babble.
2. Answer the question - David Kellogg has a great post on this key piece of advice. Executives who have limited time and patience simply want the answer to their question. If they get babble first, they will either tune out or cut you off. So listen first, then answer the question.
3. Provide supporting insight, data, and details if necessary. Once you answer the question, you then have some room to provide more information. It's likely that you will get cut off with by the next question, but if your response adds insights backed by data, they are more likely to stay engaged.

Despite having this very good advice engrained in me for almost fifteen years, it amazes me how often I get it wrong. I see it in my staff, and it's especially hard for those of us with engineering backgrounds. We're trained to think and solution, so our brains naturally jump into the details bottom up. It's important to know the audience, especially the executive audience, that are looking for simple answers first followed by insights.

Top Ten Attributes of Agile Platforms

In my last post, I stated that technology teams could achieve a 2X speed to market by leveraging agile platforms. Here are my top ten attributes of agile platforms:

• Fast and easy to learn - It lets average developers learn and be productive with short ramp up time. It has a combination of tools and documentation that lets my whole team start using it quickly.
• Built on standards - Because its easier for me to find developers with "standard" development skills - not skills proprietary to a vendor's platform. Also, because let's face it, many platforms die and if I need to migrate, I want the coding to be largely portable.
• Has an open, extendable architecture -That's a bit open ended, so here's what I mean. My vendor can't do everything! Sometimes the platform needs to be extended for proprietary needs. Also, the platform doesn't live in its own ecosystem - it probably needs to be integrated with other services, applications, and data sources. Does it have a published API that's easy to understand and leverage? Can it be deployed to cloud or virtual environments?
• Well defined performance / scalability - Because no one wants a platform that requires lots of servers for low levels of activity. "Well defined" means that the platform's architects have documented, benchmarked and, delivered tools to help developers know the performance / scalability boundaries of the platform. It should have defined approaches to monitor capacity.
• Is "easy" to install, configure, and administer - I'd like the Ops team to own the Production environments and developers need to be hands-off them. Agile platforms need the basic tools for administrators to do the basics easily and quickly.
• It must support big data - If it's easy to learn, then a development team should be able to prototype applications fairly easily. But if it can't scale to handle larger volumes of data, then these prototype applications may be no more than "tinkerware" - nice applications that can't be deployed to customers.
• Diagnostics and troubleshooting should be easy - If the development or operations team need to call the platform's Technical Support, then that's a big red flag. Good error messages, diagnostic tools, and useful logfiles are all minimal requirements.
• Well defined security model - Security capabilities often need to be evaluated prior to any prototyping, so the model and capabilities need to be well defined and easy to leverage.
• Strong development community - I certainly don't want my team to be the only ones working on this platform! Can I find a strong community? Are they happy, or bickering? Can my team turn to them if they have a question or issue? Is the best information organized into a FAQ or other knowledge tool?
• Produced "AMAZING" business results - This is really the most important attribute. Strong development teams can do quite a bit with basic commodity stacks whether it be LAMP, .Net/SQL, Java/Oracle or other similar combination. So if a new platform is being added to a portfolio, it truly must provide significant "amazing" business value. When the platform is used in applications, will it give the business a competitive edge? Will it make business teams hype efficient? Will it make the organization smarter?

I could actually think of several more attributes - but this is not a post on platform "due diligence". Feel free to comment with your suggestions.

Achieving 2X in Speed To Market Using Agile Platforms

Wenger Giant Swiss Army Knife

Developing extendable technology  platforms is what makes agile teams and agile businesses nimble. For simplicity I'm going to call a platform as any single or stack of technologies that are leveraged in more than one product or business process. Since a product or business process is somewhat fungible, what I really mean is more than one application developed for more than one target user group.

It's actually easy to see the economics. Consider all the startup costs from selecting a technology, physical installation, configuration, integration, training, and developing support processes that are all overhead above developing applications. Even if your technology is SaaS or cloud deployed, it only addresses some of these startup costs.

What is difficult to understand or estimate is the benefit when individuals and teams develop an expertise with the platform. In addition to better productivity, the team develops better expertise in guiding the business on capabilities and feasibilities. Strong teams will develop reusable components and services that factor into both speed and quality.

Achieving 2X in Speed To Market Using Agile Platforms

Let's assume you take out the overhead of selection, installation, training, etc. and only compare the time and cost to develop product #1 versus product #2? Obviously I have to assume these products are somewhat equal in size and complexity, but take a guess how much more efficient a team will be developing that second product? From my experience, it's 2x which means either they can complete the second product in half the time, or build the second product with a 2x in functionality or complexity. Yes, I have working examples from multiple points in my career leveraging different teams and agile platforms that have achieved this 2x.

This is the final ingredient in Shifting to a Market, Program, and Platform Organization. Once you know your target audiences (markets) and develop running programs using agile planning and development processes, it's the expertise around a platform of technologies that makes the team truly nimble.

Simple Agile Product Development

I firmly believe the best products are simple.

What does simple mean? Simple to use, simple to understand, simple to sell, simple to market, simple to engineer, simple to test, simple to deploy.

Simplicity comes from design patterns, reuse, and focus. User interfaces need to be easily understood. Our brains understand patterns; color schemes, design elements, navigation, messages, interoperability of core functions. Reuse simplifies construction, testing, and ongoing maintenance of the application. Focus requires a disciplined approach to prioritize the top features that deliver the highest value.

So what goes wrong?
Complexity comes from trying to do everything. Too many features implemented too quickly. Or features that provide little value. It comes from overly thinking use cases and boundary conditions instead of making things work for the 80% majority. It comes from customizations. It comes from designing intricate tightly controlled workflows. It comes from designing or developing things without consideration of standards or by developing standards when needed. It comes when quality and acceptance criteria isn't well defined. It comes from deviations in process.

Why things go wrong
There are six key people that need to be on the same page in product development. The Product Owner truly must articulate priorities and a minimal feature set to drive value. The Designer must understand or establish repeatable design elements and user interface components. The Business Analyst has to flush out requirements and drive them to patterns. The Technical Lead needs to consult on feasibility, discuss estimates, and define the application architecture. The Quality Assurance Analyst must comprehend the full scope to help define the testing approach. These five individuals often benefit from a sixth, the Project Manager, who orchestrates this conversation and has the role to drive the team to a shared understanding of the priorities, feasibility, and approach. The Project Manager needs a point of escalation if he/she believes the team isn't closing in on a shared understanding.

Things go wrong when this team is not in balance. The Product Owner may not be prioritizing (e.g. asking for everything) or sharing the rationale (e.g. the business value) behind priorities. The Designer may have conceptualized a full scoped user experience and can't easily scale back based on shifting priorities. The Business Analyst may not see the big picture and isn't help the Technical Lead to propose design patterns. The QA Analyst may have trouble keeping up with the sprint's pace. The Project Manager may be having trouble getting the team together or asking questions based on all the activity.

Making this work
These six individuals need to meet regularly. More than once/sprint. For a 2-week sprint, I would recommend as frequent as four times. They must ask questions and challenge assumptions. When the team reaches a shared understanding, then they can meet less frequently. User stories and other documentation is the byproduct of these sessions.

The team has to review the state of the product and the state of the backlog, prioritize stores, and discuss at a high level aspects of these stories. If there is an ongoing UAT (User Acceptance Testing) or Beta process, this team should review feedback. The team should also review completed stories. Are the acceptance criteria correct and sufficient? Does the estimate look accurate? Are dependencies properly accounted for in other stories?

The Simple Product

The simple product is achieved when these team members are in sync on this "simple" strategy. Target simple features that have high business value. Use "simple" as a guiding principal when considering user experience, functionality, and application architecture. Use "simple" as an  uber test for completed functionality. Keep it simple.

Being a CIO: One Year Later

Last year, I published a series on my first hundred days as CIO at McGraw-Hill Construction. I've largely been quiet since then, unfortunately (from a blogging perspective), with my head down working with the Business and IT teams. We've rolled out some new products like Dodge SpecShare Suite, upgraded several technology platforms, and established an agile project management office among many other achievements. I've also spent a lot of time meeting with customers and learning about the construction industry. I am running a CIO panel at ENR's FutureTech conference in December and started a CIO Council for construction industry CIOs (largely general contractor and design firms).

Over the last couple of months, I've contributed two articles to Engineering News Record. My first post covered Three Technologies Where Construction CIOs Need Strategies and I contributed a second this week on Tablets, Laptops and Virtual Desktops: Trends for CIOs to Watch.

Some things that I've learned along the way

• You have to focus on the now and the future - all while you are still learning. Every day I'm still learning a new detail about the construction industry, our products, customers, and our architecture - yet I still must provide guidance on projects, future initiatives, capabilities, and budgets. 
• Improving and accelerating application delivery takes both persistence and patience. Persistence to make changes to internal processes, architectures, and priorities - patience for teams and individuals to adapt.
• It's a cliche to say technology is changing - but it is in more dramatic ways. There is now a feedback loop on technology demands. Customers and users have all seen benefits in leveraging new technologies; mobile/tablets to access information, business intelligence to make smarter decisions, social media to collaborate and other improvements in usability and search technologies. Users are hooked and have higher expectations for smarter analytic capabilities delivered conveniently with better tools to make them productive.  

It's been a fun ride. More to come.