Scaling an Information Technology Organization

If your company is in the enviable position of growing rapidly and needing to scale their IT organization, there are a number of ways to expedite the process while ensuring that you get the right people on board and integrated into the team as quickly as possible.

First, we will take a look at a couple examples of what not to do. These stories are based upon first-hand experience

Avoid Accuracy through volume
It might be a good practice if you are a machine gunner, but accuracy through volume in hiring is a difficult and often disruptive undertaking. I worked for a company years ago that would hire multiple dozens of new college graduates each year. On one hand, it led to a lot of variety of thought and fresh, eager colleagues who could sometimes be a breath of fresh air at an otherwise old school and slow-moving company. On the other hand, the very high turnover and constant on-boarding where disruptive to the larger team and led work to be completed more slowly and less efficiently than it could have been with a more streamlined and well-planned team. New college hires can be a tremendous addition to the team, but they need to be balanced by experienced colleagues who can help achieve the first "what to do" item below to give the best chance of a new hire quickly becoming a productive member of the team who then stays with the company and becomes one of the experienced colleagues to keep the cycle going.

Avoid unattainable job requirements
It is very important to carefully consider what you are looking for in a candidate whether you are hiring just one or many. However, this can lead you to put together a job description with requirements so specific that no candidate will fill the bill. I sometimes like to refer to this as IT folks making it more difficult for themselves and others in IT to find a job because the IT folks are the ones building the algorithms that process resumes. When the number of requirements put into those algorithms is too high, a lot of resumes for excellent candidates can be rejected simply because of a missing keyword. There is certainly a base level of technical skill necessary to fulfill any IT role. However, I prefer to look for candidates with certain types of experiences like taking something from inception to production and an ability to passionately tell the story of interesting things they have worked on. Even a college recruit should have something they are passionate about though it might not be something technical. In all my interviews, I always ask the candidate to tell me, with a significant level of detail, about something interesting they worked on. During their answer, I pay attention to both their passion and their level of detailed knowledge on the subject. Even if the work is different from that of the role for which they are applying, a candidate who was passionate and picked up and utilized detailed knowledge in the past can do the same again for your company.

With those two examples of what not to do, we will turn our attention to a number of "what to do" items that will help find the right candidates and get them integrated into the team as quickly as possible.

Nail down your corporate culture AND your IT culture
Most companies of any significant size have a culture that is frequently carefully cultivated by HR and corporate leadership. This is definitely a great start, but there is more to be done in this area. Every company also has a culture, hopefully aligned with the overall culture, within IT. This is a culture of both soft and technical skills that needs to be cultivated by IT leadership. A few examples of key items to figure out include technology stacks, release schedules, 24/7 support, team composition, cross-team collaboration (development, operations, infrastructure, QA, etc.), knowledge sharing, colleague recognition, celebrations / postmortems, and the role of IT around the ultimate deliverables of the company. Beyond that, IT colleagues need to understand how their personal work is contributing to the success of both IT and the company. The more rapidly the IT department needs to grow, the more closely the IT culture needs to be monitored, refined, and even documented. When a candidate is partaking in an interview, the colleagues performing the interview should be able to provide a very detailed and uniform view of the culture so that all parties can determine if the candidate is a good fit. When a new hire starts, they need to be quickly ingrained into the culture to become a contributing member of the team as soon as possible. I have spoken with many people about and experienced myself instances of still not feeling like a part of the team and the IT culture even after 6 months of employment. The overall company culture may be clear, but a colleague may still have difficulty with all the nuances within IT. In the best case, such a colleague is a less productive member of the team. It the worst case, they have a ton to offer but leave to find a new opportunity where they can feel more supported as an important part of the team.

Generally speaking, many IT teams rely too heavily on HR and corporate leadership to drive the culture and forget that it is equally as important to define and refine the culture within IT and the key items that make IT unique.

Decide the right team composition in advance
Team composition to ensure the right assortment of senior and junior experience, the ratio of management to individual contributors, and diversity of thought is very important. Any guidelines can be revisited, and you certainly do not want to eliminate a great candidate who may come along at any time, but these topics need to be carefully considered and documented as you begin to scale the team.

Get the company name out there
If a company wants to attract top talent, they need to get their name out and associated with positive connotations. This can be tricky when it comes to IT if the company's primary product is not IT services. So, the company overall should ensure that it makes a positive name for itself, but leadership and key contributors from IT need to also help on the technical front. A highly technical candidate will generally prefer a company they see as a) meeting and exceeding their level of technical acumen and b) valuing technology as a significant contributor to corporate success.  The key is to give a face (more likely multiple faces) to the company at meetups and other more formal technical events. Attending and participating are fine, but sponsoring, presenting at, and even hosting will go a lot further towards cultivating the company's external, technical image. Certainly, a company needs to exercise caution to ensure that the right message is getting across to consistently highlight the best attributes. This goes back to an earlier bullet about cultural alignment above. If a company does not put themselves out there to build a reputation as an appealing place for a technologist to work, though, it will be very difficult to attract the best talent available.

Partner closely with recruiting
Assuming that your company has a recruiting department, partnering very closely with these colleagues is very important, They need to be brought into the fold of not just the corporate culture but the IT culture and the other bullet points noted in this article. I have seen many instances where IT simply creates a job description, passes it along to recruiting, and then complains that they are not getting the right candidates. Typically, a recruiter is responsible for many areas of the business or, if the company is large enough, even many areas just within IT. Working with the recruiter as a trusted partner and aligning very closely is necessary to find the right talent quickly. Beyond the job description, you need to define the personality, experiences, and other less-tangible traits for which you are looking. As candidates pass through, you need to be very thoughtful and thorough with the feedback provided to the recruiter whether the candidate was hired, offered and declined, or passed. There are many things to learn from each scenario, and it is important to share that knowledge both ways between IT and recruiting.

Work with organizations who place military veterans
This item is fairly obvious but often overlooked. Studies show that military veterans, at a higher than average rate, make for very productive and dedicated students, employees, and leaders based upon their experiences in the military. I have personally worked with a good number of veterans over the years and would not hesitate to hire more in the future.

Recruit from within
Recruiting from within comes in two flavors. One is fairly obvious and involves looking at existing employees to fill open roles. This can mean a promotion, a lateral move from one technology stack to another, or even a move from outside of IT. One of the best colleagues who ever worked for me came from outside of IT and rapidly worked his way to Director of Enterprise Architecture within fairly large IT organization. I hired him away from our original company and would be more than happy to hire him again in the future. Keep an open mind when looking within. It is a great way to keep colleagues happy through variety and challenge and to help a high performer spread their positive influence to other areas of the company.

The second flavor of recruiting from within is to mine the networks of your existing colleagues. If you are doing a good job of getting you colleagues ingrained into the culture and being happy and productive, then they should already be reaching out to friends and former co-workers to bring them into the fold. Extra incentives can but put into place to aid in this process, but I tend to disagree with significant, monetary rewards except for difficult to fill, critical roles. You do not want colleagues just using the opportunity to make a little extra money. Instead, recognition and non-monetary rewards can be used to influence participation.

Consider contract-to-hire
Sometimes companies forget about this option. Although it can be inconvenient to have new colleagues who may leave after a 6-month contract, it provides a good option for both sides to "try before they buy". If your company does not quite have the other items mentioned in this article in place, you may find really solid candidates willing to give it a try via a contract without signing up for a long term commitment. The key with these colleagues is that you should treat them just like any other member of the team because your goal is to bring them on full-time. If you isolate them from the full-time colleagues or otherwise treat them differently, the likelihood they will make the transition is greatly reduced.

7 Positive Traits of A Great Leader

In this post, I will share my thoughts on 7 positive personality and professional traits that make a great leader in no particular order. The reason I have not specified an order or numbered them is because every leader should strive for all 7. Convincing the reader to adopt some subset of the 7 is not the purpose of this article. Along the way, I will also throw in a few personal stories to highlight my points. Please note that these stories are not intended to be a statement about how "awesome" my leadership abilities are. I am simply pointing out instances where I have tried my best to embody these traits and be the best leader that I could in different circumstances. Perhaps I will write another post covering the mistakes I have also made along the way because I, being human, am certainly not perfect.

A great leader...

Practices radical transparency and honesty
You can read up on the concept of Radical Transparency on any number of websites. For the purposes of this article, I am primarily referring to the fact that a leader's job is to shield his or her team from chaos, politics, priority changes, budget snafus, etc. It is not to shield them from information. Everybody involved in a work relationship is an adult and deserves to be treated as such. No matter the information, sharing and discussing it in a controlled way is far superior to the alternative because the team is always going to find out regardless of any attempts to prevent it. Certainly there are times when a leader is simply unable to share information, and this is where the honesty also kicks in. When the team inevitably asks the leader what they know of a given situation, it might be easier to respond that they know nothing. Honesty, though, calls for the leader to let it be known that they have knowledge they are currently unable to share. He or she should explain why and then work to get to the point when they can share more as soon as possible.

A leader should both practice transparency and honesty and expect them from his or her colleagues. They must be willing to accept honest thoughts and opinions even if they differ from their own or provide constructive criticism. It is important to build a culture where such interactions are openly accepted and encouraged while also striving to be tactful in the approach (i.e. do not just call out somebody in the middle of a meeting).

I have experienced a few occasions in the past where my level of transparency and honesty have been questioned for being possibly being too "extreme". This typically comes from old-school leaders who do not believe in the same philosophy, but I can honestly say that, over 20+ years of working, I have never had an individual whose attitude, quality of work, happiness, etc. has ever been negatively impacted because I was transparent and honest with them. I have also never had another leader tell me that they wish I would provide less detail of the work my team is doing, the status of our platform, and the successes and roadblocks we have experienced because they do not want to know.

Leads from the front
A leader should never ask his or her team to do something they would be unwilling to do themselves. I am not talking about specific tasks as the leader is often not capable of executing tasks in which a member of their team specializes. I am talking about less tangible things. If a leader expects their team to work overtime, the leader should also be willing to put in the extra effort. If a leader expects their team to give up vacation time, the leader should also be willing to make the same sacrifice. If the leader expects the team to "sell" an initiative to other leaders, he or she should be right next to their team selling it and putting their own neck on the line if the initiative falters. A leader must be out ahead of his or her team doing their best push aside all the roadblocks, taking the same risks, and putting in the same effort. They cannot sit back and critique and second guess from behind.

When I worked for a large consulting company several years ago, I was in a technical and people leadership role that could have allowed me to sit back and tell people what to do while working a 9-to-5 schedule. Instead, I decided to take a very pro-active approach to the technical side by working ahead to figure out things that the team would need the next week or the next month so that I could provide guidance when we got to that point in the effort. I also helped the team execute upon all of the promises each agile sprint to ensure that everything was completed when it was an option to simply sit back and point out how specific team members had not completed their assigned tasks. This allowed me to get to know each team member, develop very solid relationships built on respect and trust, keep the clients happy, and, in the end, help to make everybody more efficient while improving work/life balance through better sprint planning. Leading from the front is definitely not the easy path, but it is the right path to take.

Is emotionally consistent
Every leader has a style, and colleagues can become accustomed to just about anything if that style is applied consistently every day. So, a leader must pick their style and stick to it versus setting up a Dr. Jekyll and Mr. Hyde situation where colleagues never know which leader is going to show up to a given meeting or interaction (in-person, phone, email). This is not to say that a leader can never make a change, and this goes back to transparency and honesty. If a leader sees an opportunity for self-improvement, he or she should be open with their team and bring them along on that journey.

More years back than I care to acknowledge, I played high school football. I still vividly remember two of the coaches in particular. All of the kids were essentially terrified of one coach because he could shift from being friendly and supportive to being a raving, cursing lunatic in a matter of moments. We never knew who was going to show up for each practice. On the other hand, there was a coach that everybody liked who was 3/4 of the way to raving lunacy at all times. We all just acclimated to his style and knew exactly what to expect and how to keep him happy. I am certainly not advocating raving lunacy as a leadership style, but the key is that people can deal with consistent behavior even if they sometimes disagree with it. At the very least, it gives the leader the ability to consciously analyze themselves and how their style impacts their team in order to execute changes for the better over time in collaboration with their team.

Exercises reasonable empathy
This is the one point (the "reasonable" qualifier) that I believe may evoke some negative feedback in this article because empathy in leadership is a very hot topic right now. Empathy is, indeed, critically important both for work and personal life, but a leader needs to draw the line so that colleagues are not taking advantage by seeing just how much they can get away with. This is not for the benefit of the leader but for the benefit of the rest of the team who are not trying to take advantage. If they see a colleague getting away with things that they should not, it negatively impacts the rest of the team. This, however, is a very delicate situation and must be approached thoughtfully and cautiously to ensure that the leader is finding the right balance.

At a previous leadership role, I was given the unenviable task of writing a review for an under-performing colleague even though I had minimal experience with that colleague at the time. I was flat out told to write a negative review so that the colleague could be put on a plan and set up for subsequent termination. Exercising a reasonable amount of empathy called for me to interview other team members with whom the colleague had worked and particularly the colleague himself to understand the situation before blindly following instructions. During this process, I learned that the colleague had been asked to execute work that he was physically incapable of executing - not out of laziness or lack of caring but a literal inability to see the computer screen the same way the average person does. He was afraid, though, to admit this and risk negative repercussions, and the rest of the team was not really aware of the situation. After learning of this, I wrote up a much more reasonable review pointing out the issue and then spent the next few months mentoring him on the back-end development that did not require the same visual abilities as front-end development. This also meant re-organizing the team to account for the change in workload, and the team was willing to exercise reasonable empathy when they understood the situation. Through the hard work this colleague was willing to put in, the understanding and trust of the team, and some assistance from me, the colleague became a much more successful developer and contributor to the team.

Decisively builds consensus
Sometimes leaders describe themselves as "consensus builders" because they see themselves as open-minded, but they often forget an important piece to the puzzle. A leader certainly needs to be open to ideas and collaborative with colleagues of all levels, but they also need to then assimilate all of that information and form a decisive action plan to move forward. This latter part is often overlooked. Just being open and collaborative does not move the needle. A leader must get all parties on the same page and moving forward together. He or she cannot expect that the colleagues with varying opinions will somehow find a cohesive direction among themselves without the leader's further involvement. A leader must dictate the direction of execution without being a dictator. The action plan may not satisfy all parties, but integrating the top ideas into a cohesive plan based upon the current situation and knowledge is the best any leader can do.

Puts the team's needs first
For a leader, putting their team's needs in front of their own can take many shapes. Perhaps it is the leader putting himself or herself into an uncomfortable situation that goes against the grain because they know it is the right thing for the team. It can also be the leader openly celebrating the accomplishments of his or her team, inviting team members to leadership meetings to present those accomplishments, and generally acknowledging the team instead of taking the successes as their own. The same can be said for failures where the leader should acknowledge their own role while helping to rectify the situation versus throwing a team member under the bus. This is a huge topic with a lot of applicable scenarios, but the key is to adopt the "we" mentality and even the word itself when discussing the team's efforts. A leader cannot always do what is best for the team, but the team should always be a top consideration in every decision.

When a start-up for which I was the VP of Engineering was acquired, an issue arose when the acquiring company sent very low offers to all of the existing employees - myself included. Because the executives of the start-up had left as part of the acquisition, that meant I was the most senior leader remaining. After discussing with the team, we came up with a plan that each person would share with me what they desired to make. I, in turn, would craft a write-up for each colleague with my honest assessment of their skills, what they contributed to the team, and what their salary should be. My portion was done privately with the trust of the team that I would be fair and honest. I then flew to the headquarters of the acquiring company and negotiated the salaries and bonuses of all my colleagues. While they did not all get exactly what they desired, each received a salary that was at least 35% higher than the initial offers with appropriate, annual bonus potentials as well. Only at that point did I negotiate my own salary. From a humanity and leadership perspective, this just seemed like the right thing to do. I worked closely with and liked all of these colleagues and wanted to see them treated fairly so that we could continue our collaboration. From a personal perspective, it also did not make sense for me to negotiate my position while the rest of the team got frustrated and possibly left for other opportunities. A leader without a team is not really a leader.

Accepts that failure is always an option
Business and technology are changing rapidly and dramatically. It is not possible to create a perfect plan and then execute flawlessly on that plan. Accepting that failures will happen and iterating upon the best ideas formed using the most up-to-date knowledge at the time are key. A leader must be willing to both accept and admit mistakes within and outside of his or her team. The key when something goes wrong is to understand where things stand and to take decisive action to improve the situation while being honest and transparent along the way. Certainly, there are more considerations if a colleague is being intentionally malicious. Typically, though, what we might call a "mistake" in business is actually a decision that was made with the best intentions that seemed right at the time but turned out, in hindsight, to be incorrect. Everyone should learn from mistakes but what is the benefit in lingering on something that, say six months ago, seemed like the best decision? Even worse, what is the benefit of trying to assign blame six months later? Simply use whatever new knowledge has come to light since that time, build a consensus for the best approach to move forward, and take action under a new plan.

Data Governance vs. Information Governance

A lot of folks have heard the terms "data governance" and "information governance", and the two terms are often used interchangeably. This, though, is a mistake as there are significant differences between the two initiatives. In this post, we will explore those differences and explain why it is important for companies to implement both.

To get started, it makes sense to spell out a few definitions. In the world of data, there is a progression from data to information to knowledge, and it is important to understand each of these.

Data is essentially just the raw data available across various sources like databases, spreadsheets, etc. For transactional purposes, it is obviously necessary as it underlies every platform out there whether it be an ERP, CRM, WMS, etc. Without the system on top of it, though, the data is not useful as you typically cannot learn anything from the data in its raw state. Data can flow among multiple systems and can be aggregated into a single location like a data lake.

Information is the cleansed and organized form of the aforementioned data. This is a very important step in the process that is often minimized by people looking in from the outside of a Data and Analytics implementation. You cannot go directly from data to knowledge without a significant amount of work spent turning the data into information. You can expect that 50% or more of the effort for the overall program will go into producing and organizing this information.

Knowledge is the set of insights gained from analyzing the information whether that be descriptive analytics where you are simply looking at what happened in the past to help you plan for the future or a fully executed neural network that is churning through huge volumes of data to prescribe actions to take. The key here is that these insights need to be actionable. Simply knowing something is not particularly useful if the knowledge cannot be acted upon to optimize or automate a process, create a new revenue channel, change a business model, etc.

This leads to my first argument for having separate data and information governance initiatives.

1) Data and information are very different from each other, and this difference is important to highlight and fully understand as it requires significant effort to move from data to information. Treating them the same tends to lessen the importance of the information in the eyes of the key stakeholders across the company while giving the very false impression that simply collecting data will lead quickly to knowledge.

Now that we understand the differences between data and information, we can take a look at who is responsible for each. Every company is different, but, generally, data governance is primarily the responsibility of IT.  IT manages the platforms and databases, creates the integrations pipelines to move data around, pulls data into analytics platforms, etc. We will hold off discussing the details of implementing a data governance initiative and save it for a future post. At a high level, though, data governance is about privacy, security, risk management, compliance, and auditing. The work involved in such an effort mainly falls into the realm of technical solutions put together by IT. Of course, there is collaboration with the business (e.g. Finance) particularly around compliance and audit, but even in those cases technical solutions are typically required.

Note:  Certain businesses like health care, banking, pharmaceuticals, etc. that tend to be very heavily driven by risk and compliance are likely to have entire teams dedicated to these areas. This might mean that a team outside of IT is actually driving those aspects of the project, but it is still appropriate to consider them part of the data governance initiative because the focus is on the data itself and not the information and knowledge that can be derived from the data.

Just about every company out there knows about the aforementioned aspects of data governance. They don't want people within the company accessing data they should not. They don't want data leaving the company and falling into the wrong hands. They must fulfill specific governmental and regulatory requirements in order to stay in business. Though not every company dedicates the time, effort, and funding required, the need is generally quite obvious.

Less obvious, though, is the need for information governance. Companies assign value to data by default when they decide that it is important to secure it. That is part of risk management - how much is it worth in dollars, reputation, etc. to have specific data stolen. However, there is also a huge amount of potential value in information that tends to go un-quantified. A lot of companies understand that they should use the data they produce to do something, but they have a difficult time building the business case and the large-scale implementation plan required to achieve what they need. This is where information governance helps. Though the initiative still requires IT support via the appropriate tools, information governance is much more business-centric. It focuses on data classification and metadata, data architecture and availability, data quality both via business and technical processes, data awareness and literacy, information life cycle management, stewardship, and value creation. As noted previously, there is some overlap with data governance, and there are certainly technical aspects typically handled by IT. However, the initiative is really about business colleagues helping to produce clean data by using their platforms correctly and then having access to that data with a high level of literacy and a sense of ownership so that they can ensure that it is of high quality and use it to create information, knowledge, and eventually value. Information governance is the framework that allows a company to assess their current state, create a plan for the future, and track progress towards the final goals. It is also a great basis for creating a business case for undertaking something like building a Data and Analytics program.

This leads to my second and third arguments for having separate data and information governance initiatives.

2) Information governance is often overlooked. Most companies have some sort of structured plan for addressing security, risk management, etc. around their data, but they frequently do not have a structured plan to address the actual use of that data to turn it into information and knowledge so that they can derive value. Splitting the initiatives ensures that ample focus can be applied to both.

3) There is a separation of duties required to focus on the appropriate aspects of governing data and governing information that can best be achieved by having IT run the data governance while folks on the business side run the information governance. Of course, there is some overlap and significant collaboration required to be successful.

Creating a Successful Data and Analytics Program - Continue to learn and refine (Part 6 of 6)

This is part 6 in a 6-part series about creating a successful Data and Analytics program. To read parts 1, 2, 3, 4, or 5, please follow the links below.

• Part 1 - Introduction
• Part 2 - Spell out your ambitions
• Part 3 - Select the organizational structure
• Part 4 - Perform a proof-of-concept
• Part 5 - Execute

And now on to part 6...

Step 5:  Continue to learn and refine

As with any complex undertaking, one must accept that 100% success is essentially impossible the first time around. Even if you've done every step in a manner you believed to be correct at the time, things will inevitably go wrong, and that's OK. Initial execution should always be followed by a period of reflecting, learning, and refining. Perhaps the business needs are not exactly as planned. Perhaps priorities must change due to customer demands. Perhaps there are technical issues that require resolution. All of these are common situations that arise after the first implementation of a Data and Analytics initiative. The key is to have a flexible platform and open mind. As I stated in previous posts, iterating on an idea and continuous improvement are not an admission of failure nor an assignment of blame. This is exactly what is supposed to happen. Gone are the days of old-school business thinking where a company needed only to focus on doing a small set of things as perfectly as possible for as long as possible. Technology, business, and culture are changing far too rapidly to get yourself all worked up over achieving perfection.

Business now is in a constant state of evolution. There is a quote commonly attributed to Charles Darwin and his research on evolution that is actually just paraphrasing his writings but still captures the essence very well. "It is not the strongest species that survive, nor the most intelligent, but the ones most responsive to change." This concept can be directly applied to modern business. No matter how strong, smart, and high-performing a company is right now, it is the company's ability to change that will make it successful in the long run. Change does not just happen at the top when setting the direction for an entire company. It must be imbued into the culture so that even something as small as iterating on a Data and Analytics project is looked upon as a positive. As long as you have spelled out your ambitions and decided what success looks like, you will get a really good idea what is on-track and what is off-track from the metrics you've designed to monitor your progress. It may even be necessary to change the criteria for success and/or the metrics. Just be sure to proceed collaboratively, honestly, and transparently. Prioritization during this phase is key. Everybody will have an opinion, and there will be so many possible avenues to take that it is very important to approach them in an organized manner to keep things manageable. The methodology for prioritization will change from company to company, but it must always align with the success criteria.

Speaking of change, the tools and techniques in the world of Data and Analytics are changing very rapidly. In my experience, this change is even faster than most other technologies. Although it is not necessary for most companies to stay on the bleeding edge, it can be easy to fall behind your competitors or become stuck with older, incompatible versions of software if you do not stay on top of things. Keep this in mind as you build out your Data and Analytics program. You don't need to latch on to every shiny object, but you do need to stay educated on the latest trends and use your success criteria to decide which will help you get closer to that success state or even adjust your idea of success to set the bar even higher.

Creating a Successful Data and Analytics Program - Execute (Part 5 of 6)

This is part 5 in a 6-part series about creating a successful Data and Analytics program. To read parts 1, 2, 3, or 4, please follow the links below.

• Part 1 - Introduction
• Part 2 - Spell out your ambitions
• Part 3 - Select the organizational structure
• Part 4 - Perform a proof-of-concept

And now on to part 5...

Step 4:  Execute

Step 4.1:  Assemble the necessities

Assuming the items identified in the prior parts of this series have been satisfied, there are 3 necessities when implementing a Data and Analytics program.

First and foremost is the data. Typically, having data is not an issue for most companies. In fact, there is often too much, and that is one of the factors that instigates the need for a Data and Analytics initiative in the first place. However, it is very important to find and understand the data and the business processes behind its creation. A common term you will hear in conjunction with this step is, “source of truth”. This refers to the fact that every single piece of data can be tracked back to the system in which it was originally created. The source of truth should be found for as much data as possible which should then be collected into a central repository in its completely raw, unmodified form. This is the foundation for all downstream analytics, reporting, and other tasks, and gathering the data in this manner ensures the ultimate flexibility in its use.

The second necessity is the platform. To house and work with the data that is collected both initially and on a continuing basis requires a powerful, flexible system. It must be able to scale up and down in terms of both storage and computing power as needs arise while serving multiple use cases as outlined previously in part 4 of this series.

The third and final necessity is the expertise. Without expertise, either in-house or external, the initiative will never take off. There are very specific skillsets, like that of a Data Scientist, Data Engineer, or Data Analyst, that are required to both implement and utilize the platform. Whenever possible, core business and data knowledge should be learned by in-house resources while tasks like configuring and maintaining the underlying infrastructure can be outsourced with minimal difficulty. Due to the popularity of Data and Analytics, skilled local, near-shore, and off-shore external resources are readily available and should be utilized appropriately to ensure the most cost-effective implementation in combination with in-house colleagues.

Step 4.2:  Horizontally expand analytics capabilities

Note:  In this section, we are assuming the implementation of a Center of Excellence (CoE) model as it is generally the model that I prefer and could work well in a majority of situations. Horizontal expansion applies in all cases but would proceed somewhat differently for each. For instance, step 1 below would be important for a fully centralized, but 2 and 3 would be less so. Though, it is still always important to have the users who will benefit from the final product highly involved in its creation. Much like getting kids to eat a proper dinner, people are more likely to use something they had a hand in creating.

When we think about horizontal expansion, perhaps one might picture their waistline around the holidays. In this instance, though, horizontal expansion refers to the introduction and adoption of analytical skills and data literacy across a wide range of the business – well beyond the confines of a CoE. This expansion of knowledge and skills involves members of IT and most facets of the business with everything done under the guidance and supervision of the CoE.

There are three levels of Data and Analytics knowledge and use that apply to this expansion. I've listed them here in order of increasing complexity.

1. Consume and take action – This capability applies to just about every colleague from senior leaders to font-line workers. All the data, analytics, reporting, forecasting models, and prescriptive suggestions are meaningful only if colleagues crafting the business plans and executing the day-to-day tasks learn to consume the reports, dashboards, and forecasts, trust them, and use them to influence their decisions.
2. Discover and share – A smaller number of colleagues will have access to modern tools like Microsoft Power BI, Tableau, etc. to discover data, analyze and create visualizations with that data, and share those visualizations with their colleagues. The CoE ensures that the tools, clean and organized data, best practices, and training are available to empower colleagues to help themselves via this top tier of self-service. The CoE can also certify what has been built to validate that they are using trusted sources of data and appropriate business logic as part of a larger Data Governance initiative.
3. Strategize and wrangle – The volume and complexity of data available to an average business today is generally more than could ever be handled by a single team. Thus, it is critical that a set of Data Champions spread throughout the organization have tools and abilities to contribute to this effort. “Strategize” refers to the act of coming up with the list of key data sources, associating those with business processes, and building and prioritizing the use cases that will best serve their area of the business. These Data Champions take ownership for the planning and delivery of these use cases even if they are not responsible for execution. The work may fall back to the CoE, an external vendor, or other colleagues, but the business must at least own the requirements and the eventual usage. “Wrangling” is a somewhat recent term in the Data and Analytics realm coined to describe the act of gathering, understanding, cleaning, and organizing data. Modern tools like Informatica, Trifacta, Paxata, etc. provide interfaces for Excel-jockeys to work in a familiar environment but with potentially terabytes or petabytes of data behind the scenes. It is important to remember that wrangling is a significant undertaking that can consume 50% of all time spent on any given use case. This effort also involves the Data Champions taking responsibility as stewards of the data that falls in their area of the business. Similar to bullet 2 above, the output of wrangling can be certified and operationalized by the CoE as part of a Data Governance initiative.

Creating a Successful Data and Analytics Program - Perform a proof-of-concept (Part 4 of 6)

This is part 4 in a 6-part series about creating a successful Data and Analytics program. To read parts 1, 2, or 3, please follow the links below.

• Part 1 - Introduction
• Part 2 - Spell out your ambitions
• Part 3 - Select the organizational structure

And now on to part 4...

Step 3:  Perform a proof-of-concept

As with most new endeavors, it is difficult to dive head-first into Data and Analytics without first implementing a scale experiment to ensure you have the correct people, technology, leadership support, and business interest. That said, a lot of companies dip their toes into analytics, find minimal value, and then slow down or back away from the initiative. In some cases, this could be a perfectly appropriate response. However, it is sometimes a reaction to a poorly conceived proof-of-concept (PoC). When planning the use case and scope for your Data and Analytics PoC, be sure to consider the following.

• Spell out your goals, document what success looks like, and develop a methodology to measure and report on progress towards that success state. Things can evolve along the way, but you need a foundation upon which to build so you are not just throwing things at the wall to see what sticks.
• Select an appropriately sized use case such that it can be completed in a reasonable amount of time. You want something significant enough to provide useful output but small enough that it is achievable in the time allotted. That time can vary, but 3 months is not unreasonable to implement something new and fairly complex with a high potential value.
• Take the opportunity to build support by selecting a use case of interest to business leadership instead of focusing solely on the technology and technical leadership. The more colleagues who have a stake in the success of the PoC, the more likely they are to contribute expertise, and the more likely the PoC is to be successful. Do not implement the PoC in secret and then spring the results on the business in an I just figured out how to do your job better approach.
• If return on investment (RoI) is an important consideration for your Data and Analytics program, select a use case with notable, calculable time saving or monetary saving/earning numbers. This is not to say that the PoC is going to directly save time or save/make money, but you should be able to clearly show how it could lead to that when put into production.
• Analyze your company's current state of data readiness and scope the use case appropriately. This can be a tough one. If your company already has a solid platform in place to handle day-to-day operational reporting and basic analytics needs, then the PoC should demonstrate the potential to do more with advanced analytics like machine learning. If your company is still struggling with the basics, a PoC that overshoots the current state by too much may not be the right choice. Such a PoC would tend to have one of two impacts. First, it could result in the question of why would the company invest in something so far-reaching before it solves for daily needs. Second, it could get colleagues and leadership excited about a future they think is just around the corner when it is actually years away and get you into an over-promise and under-deliver situation. That said, you do want to engender a degree of excitement to garner support. So, finding a balance is key.
• If you do opt for a PoC focused on forecasting, machine learning, or other advanced analytics, be sure to gather enough data to actually find actionable insights. There may be hundreds or even thousands of variables to consider before you can pare things down to a reasonable size and still provides business value. Do not go into the PoC thinking that you are going to build a machine learning model on a couple dozen variables. You are likely to be disappointed. Also, be prepared for the possibility that your first endeavor may not provide significant insights or it may just confirm commonly held beliefs. What you learn through the process is still important, and you can also pivot to something more promising. Communication and transparency is key.
• Do not miss the forest for the trees by seeing the platform as only capable of serving a single or limited set of purposes. Step back and consider the possibilities. A correctly architected Data and Analytics platform can do all manner of things. I have listed a few below, and it is important to figure out what will best speak to the leadership of your company.
• Analytical insights – Use the data to find patterns, forecast the future, and prescribe actions with the highest potential success rates.
• Process offloading – Use the massive compute power and inexpensive storage inherent to the platform to offload long-running jobs currently performed by other platforms.
• Active data archiving – Store all data, forever, on a platform where storage is cheap and the data is easily and securely available for analytics and reporting
• Real-time alerting - Process data in near real-time and produce proactive alerts when a non-conformance is found or a specific state is reached.
• Self-service – The biggest current trend in Data and Analytics is the democratization of capabilities so that users are empowered to help themselves instead of waiting in line for their needs to be prioritized and addressed
• If at all possible, use platform (PaaS) and software (SaaS) service offerings available from cloud providers like Amazon, Google, IBM, Microsoft, etc. This is particularly true in a PoC environment that you want to build out rapidly and then dispose of or possibly use for future experimentation. However, the cloud also provides significant benefits when moving to the production phase. People greatly underestimate just how difficult it is to build, maintain, and tune something like a Hadoop cluster either on premises or in cloud infrastructure (IaaS). It is extremely time-consuming and can be very expensive. Cloud services allow you to focus on satisfying business needs instead of fighting with the technology while being easy to expand as demands increase. Done correctly, the cloud can also be significantly less expensive. If your company does not currently allow it, find some partners in the organization, build a business case, and fight for it as the benefits are huge.

Speed Versus Structure In Information Technology

Since the explosion of Information Technology, it has become a necessity for companies of all types and sizes. From a small start-up to a mega-corporation, practically everybody wants to move fast and develop solutions with minimal red tape. There is a constant struggle to balance speed versus structure in most IT departments. I have worked on both ends of the spectrum. On one hand, I've entered companies who believed that they had a traditional, Waterfall methodology when what they actually had was chaos. On the other hand, I have done consulting with the most strict Agile you can find where the client, if dissatisfied with the work completed during the most recent 2-week sprint, would not pay for those two weeks. What I have discovered over the last 20+ years, though, is that there is no silver bullet. There is no single approach that will work for every situation. However, I can guarantee that there is a specific level of structure in every situation that will optimize the team and what they produce. You just need to analyze the current status and introduce structure as-appropriate while not being afraid to change and iterate until you find that sweet spot. I will provide a few of my own scenarios below in the hopes that they can help you figure out what is best for your scenario.

Scenario 1:  Hyper-strict Agile

The hyper-strict Agile scenario I noted above actually worked incredibly well. The teams I oversaw pumped out an impressive amount of very high quality code in those 2-week sprints, and our clients were very satisfied. There were risks, though.

1. It took a very special person with a somewhat unique personality and work ethic to excel under those conditions.
2. Colleagues who did not fit dropped out very quickly, but those on the borderline who maybe could have excelled if managed better had a high level of stress-induced burn-out.

I found that there were 2 keys to success in this scenario.

1. Treat all colleagues fairly, honestly, and openly while ensuring that they play a pivotal role in selecting the work to be completed each sprint. Certainly, part of the leader's job is to set high expectations, but having a team fully committed to the work before them instead of just trying to meet expectations allows for great things.
2. Lead from the front, not from the back. If you have a team you expect to work hard and put in long hours, then you must be willing to do the same. In my situation, I intentionally did not self-assign tasks in each sprint. However, I made it my mission to provide an extreme level of assistance to any colleagues falling behind and, in the second week of each sprint, to pick up every dropped task and complete them myself or in collaboration with the assigned developer. This had several positive ramifications. The first was engendering a real team spirit when they saw that the person leading them was also in the trenches with them. Equally as important, though, it gave me the opportunity to truly understand each colleague on the team, their capabilities and interests, where they excelled, and where they had opportunity for improvement. This allowed us to tweak assignments each sprint to move even faster while still maintaining quality and keeping the team happy and learning new things.

Scenario 2:  Waterfall

In both start-up companies with which I have been involved, we generally followed a Waterfall methodology with what I would call a touch of Agile. Primarily, the Agile portion was that we had one meeting a day to discuss progress, roadblocks, and plans. These scenarios were a bit unique, though, and the keys to success might not be achievable in other situations.

1. We were a small group of people all extremely committed to a single goal.
2. All team members were highly self-motivated and entrepreneurial. It was ingrained into their personalities that they were going to work hard and produce the best output they could. Plus, it didn't hurt that every task they performed essentially fed directly into their own personal success or failure due to their tight alignment with and ownership stake in the company.
3. Any addition to the team would quickly fit in per the previous 2 bullets or be replaced as we didn't worry about things like HR policies.

This has been my personal experience with highly-successful Waterfall, but I'm sure there are other situations where it can work. The key is that a lot of companies seem to refer to anything that isn't Agile as Waterfall. Per the next scenario, what they actually often have is chaos.

Scenario 3:  Chaos

Just the word chaos generally has negative connotations, and I would certainly argue that chaos is a strong negative when it comes to Information Technology. I walked into a new job a few years back and instantly recognized the telltale signs of chaos. Prior managers who had since left used to micro-manage and dole out tasks to individual team members while dissuading collaboration. Daily processes were constantly running long and/or failing. Only through the extreme efforts of key individuals on the team was everything just barely holding together. I interviewed every member of my new, large team and realized that they all had a decent idea of their own roles and the work they were doing, but had almost no idea what was going on with the rest of the team. They also all confirmed that there was clearly no process for work intake, prioritization, testing, releasing, and teamwork. In this scenario, we very quickly started to introduce some order into the chaos through structure. Certainly their were doubters both within and external to the team, but we took the time-tested approach of clearly stating our goals and developing a methodology to measure and report on our progress towards them. It is difficult for doubters in such a situation to argue with hard metrics. It did not happen overnight, be we were able to, without expanding the team, increase throughput 3-fold, improve our rate of achieving SLAs by 500%, and shrink our weekly pending tasks list (essentially bugs and non-project items) by 95% into the single digits. It took 5 months to achieve these numbers, but the initial benefits showed up very quickly. A tremendous side-effect was the overall happiness and satisfaction of the team. People even started doing their week-long on-call rotations without a single incident so they could sleep through the night which was better for everybody. All of that was made possible by finding the right amount of structure for the situation. We continued tweaking and adjusting that structure for a year through both our team organization, our procedures, and our tools (e.g. Jira) until we felt like it was just about as good as it could get.

I hope that maybe some of the information I have shared above can help you improve your own scenarios. Continuous improvement and a desire to implement change is neither an admission of failure nor an assignment of blame. It is simply a way to always be moving forward and making progress.

Creating a Successful Data and Analytics Program - Select the organizational structure (Part 3 of 6)

This is part 3 in a 6-part series about creating a successful Data and Analytics program. To read parts 1 or 2, please follow the links below.

• Part 1 - Introduction
• Part 2 - Spell out your ambitions

And now on to part 3...

Step 2:  Select the organizational structure

It is obvious from the outset that a business must create some type of organizational structure around a Data and Analytics initiative as it requires the full-time effort of at least a small group of colleagues. There are four choices when selecting which organizational structure is right for your company, and all of them can lead to successful outcomes.

1. Business unit led - When business units have distinct data sets and scale isn’t an issue, each business unit can make its own Data and Analytics decisions with limited coordination. This approach is very flexible but will typically lead to independent silos of data. This may or may not be an issue depending upon the structure of the business units within the company and their need or desire to share data. It is possible that data independence is a key driver to success. This approach can also be more expensive if multiple teams are building their own programs. That said, the cost can be minimized by at least having a central technical platform for everybody to share. Thus, it would just be the "business" portion of the program that would differ across units.
2. Business unit led with central support - Business units make their own decisions but collaborate on selected initiatives. This is a slight modification to the first approach where the business units occasionally come together on specific use cases. This collaboration is a good way to standardize certain data. Let us say, for example, that multiple business units accept customer complaints and record them. There is benefit to standardizing the codes for the set of possible complaints as well as the business processes for accepting and recording said complaints so that the metrics can be compared and contrasted across the business units. Because there is no full-time, central team identifying opportunities for and governing these collaborations, corporate leadership as well as the leaders of the various business units need to be committed to this collaboration. It is easy to miss an opportunity or for business units to diverge again over time.
3. Center of Excellence - An independent center oversees the company’s Data and Analytics while business units pursue initiatives under the CoE’s guidance and coordination. This is the middle-of-the-road approach where a smaller, central team takes on the responsibility of identifying opportunities for collaboration and governing the use cases, but the business units are also allowed a significant degree of freedom through self-service capabilities. The CoE must truly be empowered by corporate leadership to serve in this governance capacity, and they also need the right approach and mindset to build a grass-roots movement of collaboration and self-service in addition to the top-down mandate.
4. Fully centralized - The corporate center takes direct responsibility for identifying, prioritizing, and implementing all initiatives. In this case, business units are still involved but primarily to consult as subject matter experts on the data they use and their business processes. This approach allows the company to impose the highest level of governance, but it can be difficult and expensive to build a single team large enough to satisfy all of the data needs without becoming a bottleneck. Caution must also be taken to ensure that pursuing a "single source of truth" does not overshadow the actual, real-world needs of the business units. The program should always allow for some degree of governed flexibility.

The Center of Excellence approach is commonly agreed upon as the most flexible and advantageous approach with the fewest limitations. It is also statistically the most successful approach when implementing a new platform from scratch as it provides a small base of knowledge and skills while allowing for flexibility within the business units to execute according to their specific needs as skill sets are acquired and new tools and techniques adopted. This doesn't mean that the CoE is right for every company, but I tend to start there and ask, "why not?" when making the choice. One primary consideration is that the CoE approach requires business units to self-serve versus the fully centralized approach. It is important to consider if your company can make the commitment to place or train the right colleagues to serve these roles within the different business units.

In part 4 of the series, we will perform a proof-of-concept...

Creating a Successful Data and Analytics Program - Spell out your ambitions (Part 2 of 6)

This is part 2 in a 6-part series about creating a successful Data and Analytics program. To read part 1, please follow the link below.

• Part 1 - Introduction

And now on to part 2...

Step 1:  Spell out your ambitions

What will Data and Analytics provide that we cannot already do? What do we expect the ROI be in terms of cost/time savings or additional revenue? Where do we stand compared to our rivals? Will Data and Analytics provide us a strategic advantage in the marketplace or is it a requirement just to stay relevant? These questions and more are critical to consider as you embark on the Data and Analytics journey, and they need to be answered to an extent that gains significant buy-in from senior leaders. Stepping back a bit, you are, at the very least, looking to understand why you are going on this journey, what success looks like, and how you will measure your progress towards success. This applies to all significant investments a company makes, but becoming data-driven is typically such a difficult undertaking (and often expensive) that having this framework in place is even more vital.

Note:  Upper management support is not simply about money and prioritization. They, too, need to adopt Data and Analytics as part of their culture. If they expect a Manager or Director to run their operations according to specific KPIs, then the senior leadership needs to pay attention to those same KPIs and use them to review performance. If they expect others to accept and react to information gleaned from data, they have to be open to that as well even if the information runs contrary to long-held beliefs. If they want data literacy to expand beyond the Data and Analytics team, then they need to become data literate themselves and ensure that all teams are playing their role in the cultural evolution. HR, in particular, can be key to this evolution. At any medium to large company, it is not possible for a single team to achieve all of the change that is required. Significant support is necessary from leadership in all areas.

Along with answering questions like those posed above, a decision must be made on priorities. There are essentially four areas where Data and Analytics can be utilized:

• Improve existing products and service offerings
• Build new products and service offerings
• Automate and optimize internal processes
• Transform business models

Done correctly, a business can scale out their Data and Analytics initiative to cover all four bullets, but it is important to start with a narrower focus to avoid becoming overwhelmed and diluting the benefits of the initial undertaking. The fourth bullet is also, typically, a much more significant endeavor than the other three and likely not where you would want to start unless it is a necessity.

In part 3 of the series, we will select our organizational structure...

Creating a Successful Data and Analytics Program - Introduction (Part 1 of 6)

This blog post is the first in a 6-part series that outlines a framework for implementing a Data and Analytics Program. The intention is not to delve into the specific details of execution because there are many options that can work but to spell out the foundation and key principles upon which a successful program is built. The information presented is an amalgam of details culled from multiple years of research, experience, and conversations with others who have undertaken such an endeavor. The guidelines and advice are based upon industry standards and best practices along with lessons learned and the latest emerging technologies and techniques.

Introduction

The first thing to recognize and accept is that Data and Analytics is not an IT or even a technology initiative. It is a business initiative that requires technical know-how and support. The concept of Data and Analytics needs to be deeply embedded in the organization at all levels so that information and insights are actively sought, shared, and acted upon. Focusing solely on the technologies involved and centralizing the initiative within IT without significant collaboration are two very common reasons given for why Data and Analytics programs fail. That said, this 6-part series is not going to be yet another "Top 10 Reasons Why Things Fail" story. Instead, we will look at how a company can implement a successful Data and Analytics program and the roles that various colleagues play.

In part 2 of the series, we will start by spelling out our ambitions...

Back to Blogging

After working for the last several years either in a start-up with 100 hour work weeks or for a couple of companies with significant restrictions on their employees' public image and information sharing, I am once again in a position to get back to blogging. Though I know this blog isn't particularly popular, it is a great creative outlet for myself and allows me to really align my thoughts. If something I write can also help at least one other person in their career or with a personal technical challenge, then I am very happy for it.

On The Importance of Code Reviews

From my "I wrote this a quite a while ago and never got around to posting it" collection...

I recently reviewed some code that contained a number of shared, global variables. Basically, a shared variable is re-used by all callers to the particular program in which it is defined. If one caller sets the value to “foo”, every other caller to that program will see the value “foo”. You could think of it as a chunk of shared memory that can be accessed by any instance of the program in which the variable is defined. A real world analogy might be an office bulletin board where every worker has access to read and post important news. That works fine if only one or two people are involved in changing the items posted to the bulletin board at any one time, but what if 100, 1000, or 10,000 people all tried to update the posted news items at the same time?

The shared variable feature of VB.NET can be used to great advantage because it allows us to perform long-running tasks only once and share the results across every caller. Thus, all but the very first execution will run much faster than they would if the long-running task had to be performed every time. That said, we also need to be very careful when implementing this technique (search for SyncLock for more information).

In the code I reviewed, every execution of the program was changing the values of the various shared, global variables, and this introduces a major issue. The problem is that every caller has the potential of overwriting the data of every other caller at points in time when this will cause errors or erroneous information to be returned. Due to the way applications are typically tested prior to reaching a production environment (very few users performing any given task), it is unlikely that such an issue would be found prior to it impacting end users. That is, unless you hold a code review attended by people that can find such a flaw and help come up with a solution. If the code in question had not been reviewed, I can say without a trace of exaggeration that the result would have been weeks to months of customer complaints about strange, intermittent errors. Numerous hours and dollars could have been wasted trying to track down and fix the problem. Instead, the problem was resolved prior to ever making it outside the development environment. This is just one reason why code reviews, when done properly and involving the appropriate participants, are an invaluable part of the development process. Every project should allow time in the schedule to both hold and react to the comments from code reviews.

It is important to note that a code review does not always have to be what one might typically envision (multiple people locked in a room for 60-90 minutes). The idea is just that every piece of code we write has been seen by at least one more set of knowledgeable eyes prior to being thrust upon the users. The number of reviewers should scale up for code that is more critical, complicated, or involves the security of a user's account and/or private information.

How to Get Your iPhone Photos to Sort Chronologically

Please note that I am using iTunes running on Windows Vista to sync photos onto my iPhone 3GS. Just a heads-up. Most notes will still apply to other setups.

I don't know about the rest of you, but I prefer all my photos to be sorted in chronological order. This is particularly true when it comes to my iPhone. I don't usually sit and watch a slide show or even flip through multiple pictures. What I often do, though, is think of a picture I want to show somebody and scan through my photo library (several hundred photos) until I find it. Having every picture sorted by the date it was taken makes this task much easier. I won't go into the intricacies of how iTunes decides to order your photos when putting them on the phone. I've done hours of experimenting, and the easiest thing to say is that it tends to sort of try to keep them in whatever order you sorted the folder the last time you looked at it in your Windows folder browser. I'm being intentionally vague here because that seems to be howApple decided to implement this feature. Whatever it really does (random number generation, contacting Steve Jobs behind the scenes for input, etc.) makes no sense at all, but it must be some strange interaction between iTunes and Vista.

Anyways, the solution I use is the following. Since I organize my photos with Google Picasa, I simply add all the pictures I want to put on my iPhone to an album for that purpose. Then, I select and export them to a single folder. During this process, I opt to shrink the photos down to a smaller size to conserve space. Regardless, these steps aren't that important. You just need to get all the photos you want on your phone into a single folder. Then, I rename the files with the date and time they were taken so that, when sorted by name (Windows default), they are in chronological order.

Of course, I don't sit there for hours renaming. There wouldn't be much point in blogging about that. I found a free program called Siren that can do it for me. Siren has many, many features and can be a little confusing to use at times, but I'll give you a simple command-line that you can execute. Let's assume that your photos are all .jpg and reside in a folder called "C:\Temp\iPhone Photos". Note that the folder should contain a COPY of your photos and NOT THE ORIGINALS just in case something goes wrong. You can run the following from the command line:

"C:\Program Files\Siren\Siren.exe" /D "C:\Temp\iPhone Photos" /N /E "%%Xdo.jpg" /S "*.jpg" /R /Q

It looks complicated, but all it does is execute Siren on the contents of the folder with a few key options enabled. Each .jpg file within will be renamed with the date and time (24-hour) the photo was taken. For example, a photo take August 2nd, 2009 at 8:41:46 AM would become "20090802_084146.jpg". Note that the date is in kind of a funky order (year, month, day) depending upon what you're used to. This is just so that Windows and iTunes/iPhone will sort it properly. That's all there is to it. You now have a folder of photos named in chronological order just waiting to be synchronized to your iPhone. You can add more photos later and run the program again.

PS - In the off chance that you somehow have two photos taken at the exact same second, I would just use a free program like Exifer to change the date/time metadata of one of the photos to be a second earlier or later.

Woodworking

Two posts on the same day! I just wanted to share some of my woodworking project photos available on Flickr (http://www.flickr.com/photos/cmaterick/sets/72157603138071980/). For whatever reasons, I just never adjusted the security settings to open these to the public as all my photos are limited to friends and family viewing. I also have a couple of projects in the works. So, check back in if you're interested.

This is the home theater I built in my previous house a few years back.



Craig

Finally on CodePlex

As demonstrated by my lack of blogging this past year, things have been pretty busy. I suppose that's just what happens when you've got a job, wife and kids, a house with things to fix, and a mild obsession with home automation and audio distribution that eats up all your free time.

This post is just to mention that, of all the little .NET programs I've written over the past few years, I finally have one that I think may really be useful to other folks. It's been my goal to get an open source project up on CodePlex, and now I have (http://www.codeplex.com/MediaManipulator). The name, MediaManipulator, isn't all that creative, I know. However, the tool is quite useful. It can do a couple different things including embedding your hi-resolution cover art from iTunes into your audio files (while alerting you of issues like missing music and artwork) and exporting playlists from iTunes while sorting the contents according to your specifications (artist, song name, etc.). Please check it out and let me know what you think. The code is tidy, but there's not a ton of commenting at this point. I have a few updates in mind for the next time I find a couple free hours.

Craig

Save 8% or More on eBay Purchases

NOTE: Since the % cash back you get it based-upon the keyword you use and your timing, I've written an application that puts the latest, highest % keyword into a twitter feed available at www.twitter.com/wulgaru. Look there to get the best keyword to use in the steps below.

For those who haven't used it, there is a site called Jellyfish through which you can get cash back for shopping at particular stores. Microsoft actually purchased them and turned it into the Microsoft Cashback program integrated with Microsoft Live Search. They are currently offering the following deal that gets you at least 8%, up to $200, on tons of ebay items. You can also use the offer up to 12 times for a maximum savings of $2400 reimbursed directly to your PayPal account!

• Go to https://cashbackaccount.search.live.com/cashback/welcome.aspx and set up a Microsoft CashBack account (just sign in with your Live ID and tell them your address).
• Go to search.live.com
• Search for the keyword noted in bold purple at the top of this post to trigger a % off ebay offer (8% and up). Note you don't have to buy what you search for on Live. Once you get to ebay, you can search for and buy whatever you like.
  
• If the offer is triggered, you will see a gray box that looks something like this. You may have to search multiple times to trigger the offer.
  

• One or more of the links will take you to ebay (pick the one that has the highest % off) where you then have a 60 minute window to find and buy whatever product you like. As previously noted, your % off coupon has been triggered and you can now search for anything you like in ebay. Technically, you can find the product in advance and add it to your watch list instead of trying to beat the 60 minute clock. You need to use Buy It Now and pay via PayPal. On the page where you confirm your purchase after clicking the Buy It Now button, you will see how much cash back you are going to get (look because there are special instructions for some sellers). Don't worry, though, because you haven't agreed to purchase the item yet when you're on that screen. Confirm the purchase, check out, and follow the directions (you will need to click a link to Microsoft's CashBack site at the end to claim your cash back).
• If the item has a return policy specified (in the return policy section of the listing not in the description), your cash back will become available 60 days later (they wait until you cannot return the item) in your Microsoft CashBack account and be automatically transferred to your PayPal account. If the item does not have a return policy, you get the cash back instantly in your PayPal account! Please note that I cannot guarantee this last statement if it's your first purchase using the offer as they may have a different rule for that. I have purchased 4 items so far and have received instant cash back on two of them (#2 and #4) ostensibly because they had no return policy.
  

So, you may get it instantly or you may have to wait a little while for the $, but 8-35% off a whole bunch of stuff is nice. Plus, your pending cash back shows up right away in your Microsoft CashBack account, so you're not wondering whether or not it worked. The cash back limits are $200 per purchase and up to 12 purchases per account. To make a second, third, etc. purchase, simply go back to live.search.com and start over with a new search. Here are the official details.
Please note that nobody appears to know when this offer will end, but I can't imagine it will last beyond the holiday shopping season.

EDIT: Added details about instant cash back and being able to search for anything once you get to ebay regardless of what you search for in Microsoft Live Search. Added Magellan GPS search term. Added note about the keyword not being important but the timing triggering the different % savings.

EDIT: I made purchase number 12 with this offer. Upon attempting # 13, I was informed on the eBay checkout screen that I had already exceeded my lifetime limit for the deal.

Microsoft PDC 2008 - Better Late Than Never

By "Better Late Than Never, " I'm referring both to my delayed post and the fact that PDC was cancelled last year and rescheduled for this year. For those who don't know, PDC is the Microsoft Professional Developers Conference that was held in Los Angeles at the end of October (10/26 - 10/30). It is scheduled "as-needed" and provides Microsoft with a stage to unveil new platforms, tools, applications, etc. to a few thousand tech geeks and a bunch of press folks. I was mainly Twittering about it this year, so I'm a little behind on my blogging. Here's a recap of some of the main items introduced at the conference this year. Some of the information is a little fuzzy as the final workings of the technologies were not made 100% clear. It's possible that even Microsoft folks are a little unsure as well.

Microsoft Azure

Azure was the major unveiling for the conference. It is Microsoft's cloud computing platform and is somewhat similar to Amazon's EC2 with a Microsoft spin. Basically, Microsoft is ramping-up their data centers around the US (eventually around the world) to house tens if not hundreds of thousands of servers upon which Azure versions of all of their business platforms will be available. These include Live Services, .NET Services, SQL Services, Microsoft Dynamics CRM Services, and Microsoft SharePoint Services. You can then write applications that run locally and take advantage of any or all of these services running in Azure. Alternatively, you could host your entire application in the Azure cloud and not maintain local servers. The usage fees were not spelled-out, but there was reference made to each application being able to "expand" and "contract" automatically to satisfy user demand.

Windows 7

Although Windows 7 was already in the news, PDC was the first public unveiling of the Vista successor. Microsoft claims to have gone through every piece of Vista with a fine-tooth comb, from the kernel to the user interface, and tweaked just about everything. Here are some noteable items that were demonstrated during the keynote and subsequent classes.

• The architecture for Windows 7 has been completely modularized allowing for custom versions to be easily assembled. For instance, Microsoft is planning to release a slimmed-down edition for netbook computers. Even the full version, with all the bells and whistles, requires far less horsepower to run than Vista.
  • I'm planning to install the pre-beta release of Windows 7 I received at the conference on an Acer Aspire One netbook with a 1.5GHz Atom processor and 1GB of RAM (once the PC arrives at my house this afternoon). To prove a point, Steven Sinofsky, the head of Windows development, has been running Windows 7 on a netbook with even lower specs as his primary PC for the last couple months.
• Development is being synchronized with Windows Server 2008 R2 so that the two work in harmony with some of the new features.
• The User Account Control (all those warning pop-ups) has been toned-down. It became clear that users either disabled UAC in Vista or became numb to it and approved everything. In Windows 7, it will be less annoying and customizable.
• The Taskbar has been replaced by the Superbar. If you're familiar with Mac OS X, you will notice many similarities, but MS has taken things a bit further. Instead of trying to explain everything here, this site has a decent video that shows the Superbar along with the new Jumplist and Aero Peek features. Technically, there's even more to Aero Peek that allows you to get a full-size preview of the window as it would open on the desktop were you to click on one of the thumbnails. It also allows you to temporarily make a window transparent so that you can "peek" at what's behind it.
• Aero shake is a very unusual feature that allows you to grab a window and shake it to minimize all other windows. Shaking the one window again will bring the others back
• Notes for many more features are available online, but these are the main ones that were demonstrated.

Windows Server 2008 R2

A really interesting feature was introduced for both Windows 7 and Windows Server 2008. I'm including it here to make it stand out from the big list of Windows 7 items. Basically, Microsoft claimed that you will be able to boot a physical computer (destop or server) from a virtual PC instance that you've mounted as a mapped drive. It's a little difficult to explain what this means, but I'll try. Let's say you have a single server that's running several virtual servers and you're hosting a website that gets really busy at Christmas. To satisfy the demand, you want to set up an extra, physical server and dedicate it to the application (no virtualization). The idea is that instead of spending the time to install the application on the new server, you can simply put a copy of the virtual PC image on the server and then cause that server to boot to the OS, applications, etc. that are on the virtual PC image without actually running the virtual PC itself. It's as if the virtual PC image has become the C drive for that server. The same can be said for desktop applications. At the conference, we got a virtual PC image in which to test a beta build of Visual Studio 10. I can put that image on my netbook after installing Windows 7 and actually set the netbook to boot to the OS encapsulated within the virtual PC instead of booting to the typical OS and then running the virtual PC on top of this. Of course, this assumes that the feature is enabled in the Windows 7 build I have, but I'll know soon.

Live Mesh

Live Mesh is a synchronization framework that allows data to be shared across multiple devices (Windows, Mac, mobile). The Beta for this is actually available now. You can go to the Live Mesh site, sign up, register various machines, install the software, and set up folders to automatically synchronize across all of them and your online desktop. There is also an API for synchronizing application data. Read more about this in the Office 14 section below.

Visual Studio 10

Not too much was said about Visual Studio 10. Perhaps that is why they've already announced PDC 2009. Having two back-to-back is unusual since they are typically held at least 2 years apart. The only item of note that I have is that the new version of Visual Studio is actually built as a Windows Presentation Foundation application. It's good to see Microsoft really using its own tools. I believe that future versions of Office and other applications will also use WPF and .NET.

Office 14

Microsoft announced that mobile and online versions of Word, Excel, PowerPoint, and OneNote will be made available at the same time as the next desktop version of Office. The online versions are supposed to be cross-browser/cross-platform compatible. Also, all versions (local, online, and mobile) will be built using Live Mesh to allow document synchronization. They gave a demonstration of multiple people simultaneously working on the same document from a local copy, online, and via a mobile phone. They were able to collaborate and see eachothers' changes. For personal use, there will likely even be free versions of Office online available similar to Google Docs, Google Spreadsheets, etc., but the experience will be very similar to the desktop versions.

A really interesting feature is that corporate customers wishing to use Office online will have the ability to license and host their own versions (for data security). Depending upon the size of the company, this option could be significantly cheaper than buying licenses for desktop copies of Office for every employee while still being secure.

Surface

There were some interesting demos for the Microsoft Surface (table-top) computers both during key notes and spread throughout the conference. It's a very fun technology to play with, but I don't see too many practical applications yet. One keynote demo, however, showed a very fascinating take on the surface that allowed to to see one thing on the surface screen and something completely different on a screen held above the surface. I believe that was on Wednesday morning, and it's worth watching.

ASP.NET AJAX Future

Microsoft is doing some very interesting things with their next release of ASP.NET AJAX. Without going into too much detail because nothing is finalized yet, they are basically adding a bunch of functionality that you would normally only find on the server end. This includes event handling, data binding, and more but written in such a way that it is completely compliant with XHTML and JavaScript standards. It's a little unusual to look at, but it functions simlar to .NET code on the server side. You create a control in HTML, use JavaScript to bind a datasource to it, and ASP.NET AJAX takes care of the rest (like requesting the data from the server, receiving the data back in JSON format, and then rendering the data as browser-compliant HTML). You can find out more at the ASP.NET AJAX website.

Microsoft Boku

For me, this was probably the most interesting keynote demo. Boku is a project from Microsoft Research that should be available early next year. It's basically a way to get people of all ages (particularly kids) interested in computer programming by allowing them to build their own games using a very clever, icon-based system that is all controlled using a gamepad (on a PC or an XBox 360). You really have to see the videos on the Boku site to understand, but I'm already looking forward to spending some time using it with my daughter so that she has a better understanding of computers and what her Dad does for a living. The depth of what you can build very easily without even using a keyboard is amazing.

Tech-Ed 2008

Last year around this time, I was posting a daily blog with photos and thoughts on the Microsoft Tech-Ed 2007 conference. This year, I'm at my last day of the Microsoft Tech-Ed Developers 2008 conference. I was so exhausted last year, that I skipped the daily blogging this year and took things easy in the evenings. However, I thought I would put up at least one post containing my thoughts on this year's extravaganza.

I'll start by saying that it wasn't the extravaganza that it has been in past years. They split the conference over two weeks, so there's only 6000 or so people here this week (versus 14,000 or so last year). That, perhaps combined with the general state of the economy, has lead to lots of very apparent cut-backs. Little things like the empty tables that used to be piled high with snacks and the lack of any drink choices beyond pop and coffee, while not the end of the world, were definitely noticeable. I also missed having the option of attending a couple of the IT Pro sessions to get a different perspective on things. There were only maybe 1/3 of the usual vendors on site, and none were giving away big prizes. Last year, there were thousands of dollars, a Ducati motorcycle, and more. The party at Universal Studios was OK, but it lacked the same spark as last year's event at Islands of Adventure. For one, Microsoft had only rented-out half of the park. Several rides, including The Revenge of the Mummy roller coaster (my favorite) were off-limits to us. On a side note, I have to point out that Universal calling the Simpsons ride "new" is quite an overstatement. They just put a paint job on the old Back To The Future ride, replaced the Deloreons with roller coaster cars, and changed the movie that plays on the screen. Other than that, it's the same ride. It would be like taking your car with 50,000 miles on it, painting it and installing a new exhaust, then telling everybody its a "new" car. But I digress... :)


That said, the real point of attending the conference was still of similar caliber to years gone by. There was a ton of stuff to learn and a bunch of really good (some not so much) sessions from which to choose. It still got me jazzed about all the latest Microsoft offerings just like it always does. If this had been my first Tech-Ed, I would have been overwhelmed and wowed just like I was several years ago when it WAS my first Tech-Ed. However, the feel just wasn't the same this year. All things considered, I'm sure I'll be back again. My guess is that the Developer week of the show will continue to grow in size and I will, once again, be wowed.

PS - I almost forgot to add my usual plea to Microsoft. I submit this same request as part of every evaluation for every session.... "Slides should be made available prior to presentations to facilitate note-taking particularly by folks with tablet PCs. I've been asking for this for years, and I just can't understand why it never happens. If the slides are required in advance of the conference, why can't they be made available before the sessions? If you're looking to sell tablet PCs and copies of OneNote or Office with OneNote, this would be a perfect way to advertise their functionality. Print the PowerPoint to OneNote and write your thoughts right on the slides as the presenter is talking. It gives your notes much more context when you go to review them after the conference."