This is from an AMA (Ask Me Anything) session that I conducted on LinkedIn on November 28, 2022. The premise of the AMA was for people to ask me anything about the world of Fractional CTOs or anything else that they could think of.

Arun Aravamudhan

Great answers. A couple of questions: (1) Given that you are billed by the hour, how do you balance recommending long-term / irreversible decisions (eg tech stack or framework decisions) vs short-term tactical? (2) What happens if you get the sense that the existing team doesn’t have the skillset to accomplish what you have set out to do? Do you bring yr own team on board (example: SREs, java/ reactjs developers etc)?

Marc Adler

Thanks for the questions, Arun.

> Given that you are billed by the hour, how do you balance recommending long-term / irreversible decisions (eg tech stack or framework decisions) vs short-term tactical?

This is why I like to get new clients when they have not started any development yet. Because I get to make the technical decisions. And these decisions are not affected by the fact that I am billing by the hour. These are technical decisions that try to carry as little risk as possible, and I always try to stick to well-known, well-tested platforms and patterns.

Your question is interesting because I try not to choose technologies that I will need to be around for a long time in order to make sure that I build a team that knows how to make use of these technologies effectively. For example, I might not suggest … let’s say … quantum computing as a technology to use because (1) I don’t know it and I would have to spend significant time learning it, maybe at the client’s expense, and (2) it’s hard to hire a remote dev shop who has people with these skills. So I try not to lead a founder and a dev team down a path that they cannot recover from.

> What happens if you get the sense that the existing team doesn’t have the skill set to accomplish what you have set out to do? Do you bring yr own team on board (for example: SREs, java/ reactjs developers, etc)?

There have been many times when a member of a remote dev team does not work out on a project that I am leading. This is why I like to use dev shops that have a lot of people. Because there is usually a variety of specialists that float around those companies and can be brought into a project on a temporary basis.

However, I will sometimes bring on a specialist that I might know if I cannot find someone with that skillset in the remote dev shop.

Arun Aravamudhan

Marc Adler

Fantastic answers. I love your take on ‘well-known, well-tested platforms and patterns. Boring is better (straightforward and solves the business problem). Example: modern Java as opposed to GoLang for a (boring) web app that needs to scale (resume-driven development wink, wink).

Mark Batty

As the AMA is still open 😉 I’m curious if/how you handle trunk-based dev with large features, feature flags, or something else for example?

More importantly, as a fractional rather than full-time how do you create/develop environments that encourage best-practice/high performance?

Marc Adler

Hi Mark. Yes, the AMA is still open. It’s a living and breathing document, and people can feel to add questions here whenever they want.

Concerning creating the dev environments … I try to suggest my own dev standards to the remote dev team. There are a number of standard things in my toolbox that I have accumulated over the years and I feel most comfortable with that toolset. This includes things like proper source code control management, CI/CD with gated checks, a complete set of automated tests, etc. But I like to try to have that one really senior dev leader on the team who will be the leader from the remote dev shop’s side. And sometimes, these remote dev shops have their own best practices for development. As a fractional CTO (as opposed to a full-time CTO), someones I let the dev team use their best practices once I get to know them and understand them. Being a fractional CTO on limited hours for my client, there is only so much time that you can put into creating the perfect dev environment. Sometimes, you just have to give the team guidelines and loosely enforce them.

And, yes … I do like to use feature flags and various other architectural patterns that make it easy to enable/disable certain capabilities as the app is being tested. I also like to have apps use dynamic configuration, so settings can be changed at run-time.

All this said, I am a lifelong learner and I am always happy to get advice, learn about new tools and frameworks, and overall, make the dev teams as productive as possible.

Mark Batty

Marc Adler thanks, all good points, and lifelong learning is definitely the key.

Those of us with 30+ years of experience know a lot, have done a lot, and have seen a lot – but not absolutely 

Richard Meyer 

Hi Marc, here’s a question: what’s a sustainable ratio of QA engineers to developers? ie: people who either run manual or automated tests. Obviously, this is a bit of a “how long’s a piece of string” question, but what’s a typical industry standard ratio?

Marc Adler

For the teams that I like to put together, the developers should be writing the unit tests (yeah, good luck with that!). But the number of AUTOMATED QA engineers depends on the number of use cases and the types of platforms the app is designed to run on (web, iOS, Android, headless). For automated tests on mobile devices, you might need some specialists who know those platforms. But generally, I don’t think of ratios of QA to devs. I think of how many automated QA engineers it would take to fully implement tests for all of the use cases in the BRD. Both positive tests and negative tests. Of course, you also need to think about Soak and Stress testing, which might require a specialist and another testing framework.

Scott Davis, Computer Software Executive (retired) & Board Member

Hi Marc,

In my experience, CTO can be a bit of a Jack of All trades role, at times a combination of Chief Architect and/or Engineering leader, technical visionary, product champion, and definer, lead customer engineer for sales, technical marketing, business development and partnerships, technical spokesperson for the company including being the person who convinces customer’s to buy and investors to invest, and ensuring that the engineering team is kept apprised of relevant new technologies and products, just to name just a few! It’s very clear to me how it might be very productive for a startup to engage a fractional CTO for the Chief Architect/VP of Engineering type role, but less so for all the other potential duties. Would like to hear your perspective on these other duties and when a fractional CTO is appropriate or not. Thanks!

Marc Adler

Wow, the former CTO of VMWare asked me a question. I am honored!

All of the various responsibilities that you listed are things that I have done at one time or another, but it really depends on the kind of company that you are the CTO/CA of. For example, for large financial companies, I really do not have any interaction with customers, unless you consider the traders and portfolio managers my “customers”. But, in those companies, I have been responsible for most of the other functions that you listed.

Since many of my CTO-as-a-Service clients are startups, they don’t really have customer bases, and I don’t really have to keep the engineering team apprised of new technologies, since the technical teams are often offshore developers whose sole job is to implement the product. Of course, while I am involved with my clients, I keep my eye on interesting technologies, but I have to be very aware that most of my startup clients have just enough funding to build an MVP and at the early stages, they cannot afford products that have huge license fees.

As far as “technical marketing” goes, I will often be responsible for the technical parts of decks that are presented to potential investors and customers. I will also meet potential investors at times.

If you want my opinion about the most important job of the fractional CTO, it’s to represent all of the interests of the startup founder when dealing with development teams. Every week, I speak to one or more founders who mocked up their idea in something like Figma and threw it over the fence to a development company, leaving all architectural and technical decisions to the development company. Many times, the startup founder has zero ideas of what the development team is doing, what infrastructure they have chosen, what future costs will look like, etc. Sometimes, the founders don’t even know where their source code is being kept. I have said this over and over … a non-technical founder should NEVER go into an engagement with a development company without a senior technical person on their side, representing their interests. This is a function of an fCTO that is really not applicable if you are the CTO of an established company.

Of course, I do all of the technical stuff for my founders, including the architecture, cloud setup and administration, running the dev team, helping to write the product requirements, etc. In a startup, the fCTO often combines multiple functions that, in established companies, you would have separate individuals doing.

Sam Alexander

How do you handle “chicken vs. egg” prospective clients who are looking for a fractional CTO to attach to their venture in order to secure more funding – so that they can hire a fractional CTO? To what extent do you offer assistance (free or paid) before there is funding for the full MVP/project (ie. giving budget ranges, looking over pitch decks, giving them a bio to share, etc)?

Kathy Keating

I will typically give 1-2 hours of conversation for free, then we will work out a plan together based on what the founder/company can afford and what my pay rate is.

I am a Techstars mentor for several of their accelerators (and globally). My time is offered freely to founders in that network when they are in the program. I’ve been known to create a small number of equity arrangements in exchange for advisory services for founders who are out of the program but don’t yet have traction enough to pay for my services.

As Marc Adler says, each fCTO has a slightly different agreement. I tend to operate with a monthly retainer that includes a set number of days worked, on average, per week and access to me via Slack.

Marc Adler

Unlike Kathy, I do not work with equity arrangements. I have gotten burned by a few of those in the past, and at this point in my career, cash is king. I get solicited all of the time from people who might find me on AngelList or through my website, wanting me to be their co-founder in exchange for equity. Although that might appeal to certain fCTOs, it’s something that I just cannot do.

Mark Batty

Hi Marc Adler, as a (full-time/employed) CTO we sometimes work with partners to provide various specialized/expert products/services.

I’m curious if you work with any partners (formal or informal) and if so what are the differences when fractional?

Also any tips you might have; particularly around partnership agreements, fees, etc.; thanks.

Marc Adler

Hi again, Mark. I have a few dev shops that I work with. Since my job is fractional, and it involves multiple clients, my “surface area” is spread over multiple projects at a dev company. Sometimes, this will give me a bit of leverage in terms of getting good prices for my clients. It also sometimes lets me have an easier path when disputes happen.

As an example, let’s say that DevCompany X wants to charge my client $50/hr for a mid-level backend developer. I might say to them “You charged my other client $45 two months ago, and I told my new client what the expected rate would be.” So, because I have multiple clients, I have a lot more recent history and knowledge, and I can use that to help my clients.

That being said, I am not tied into any formal arrangement with any company over the span of multiple clients. If there is another company that can provide an equal or better service to a new client, then I am happy to explore those other possibilities. It’s all about serving the client in the best way that I can.

I also want to stress that I have no financial ties with any of the dev companies. Maybe I will get a box of chocolates for Christmas, but that’s about it.

Mark Batty

Marc Adler thanks, that helps.

Janani Vasudevan

Marc Adler – thank you for doing this. When someone wants to transition to being a first-time CTO, what are some things you’d ask them about their readiness? Wondering what the main differences with being a VP or director of engineering and transitioning to be a CTO

Marc Adler

Hi Janani. Thanks for this very interesting question.

First of all, as I have mentioned, the title of “CTO” takes on many shapes and sizes. You can be a strictly high-level, non-technical CTO … I have seen many CTOs that were really Product Managers … or you can be the senior developer on a two-person dev team … I have seen those as well. It also depends on the organization that you are shooting for … do you want to be the CTO of a mega-corporation or the CTO of a startup? Each situation demands different skills.

I view the CTO position as being more strategic and high-level than the typical VPE position. Whereas a VPE or Director of Engineering might concentrate on getting the features built and keeping the system running smoothly, a CTO is generally focused on more of the planning, budgeting, and exploration. The CTO will also typically face off with CTOs and CEOs from other vendors as partners, will interact with the CIOs and CEOs, and might be present at the executive level and board meetings. The CTO is responsible for every technical aspect of the organization, whereas the VPE is concentrating on the more tactical stuff.

What I said above is definitely a huge generalization, and again, it depends on the kind and the size of the company that you work for.

I am going to point you to an article that I wrote three years ago on the role of the Chief Architect. In some organizations, the Chief Architect is the de facto CTO.

Thoughts about the role of Chief Architect

Janani Vasudevan

Marc Adler thank you!

Vinicius 💻⛰ Gravina da Rocha

Curious about your thoughts on NoCode

Marc Adler

I will be honest with you … I have not kept up with the evolution of some of the modern NoCode platforms like Bubble. I know several founders who created their MVPs using Bubble, and some of those MVPs were sufficient to secure more funding. But, when it came time to build the actual application, most of them moved away from Bubble.

Also, using Bubble required a non-technical founder to spend time learning the platform rather than spending time on things that founders should be doing. Maybe this is why there is a healthy cottage industry for freelance Bubble consultants.

Someone else is going to have to comment about the viability of the current version of Bubble to build a production application that is resilient and can scale.

Vinicius 💻⛰ Gravina da Rocha

What stack would you choose for a web app and why? (As a thought exercise, that’s the requirement: a web app.. hehe)

Marc Adler

I have to admit that I have always been a big fan of ASP.NET for backend work. Although I also love the Node ecosystem, I have been involved with Microsoft technologies for as long as I remember. And, React for the front end is usually my choice. A simple web app with some amount of API support can easily be done with ASP.NET and React.

Of course, the choice of technologies ALWAYS depends on the requirements of the app. I would never choose any technology without first completely understanding what needs to be built.

Like

John Schnipkoweit

Thanks for doing this AMA Marc, long-time follower, first-time caller.

I saw your mention about not generally coding as an fCTO – which I agree with and generally steer away from – however, on occasion, I have been able to successfully crank out a v0.1 for the right client (and sometimes to just scratch a personal coding itch). When I have done this, I have found it to be an accelerant at learning what actually needs to get built (not surprisingly) and as a result, key to building the right team. Since it sounds like you have worked with zero-to-one startups, have you found a method that works well for teams who have no engineers and limited resources to get a 0.1 version built? Do you bring in known agencies, maybe work in parallel with an accelerator, or bring on contracted engineers? Having done all of those, I understand it could just be specific to the company, but I’m curious if you have found a repeatable pattern that has created a preference for you. Thanks!

Marc Adler

Thanks for the questions, John.

If a startup can afford it, I can code up the initial skeleton of the app. I have been lucky to have been involved with some founders who have been very well-funded, and that gives me the cushion to do some coding. For most of my founders, my hourly rate is too cost-prohibitive for me to do any kind of coding.

When I had version 1 of CaaS, there were a few development firms that I used to go to. The agencies were known, and I personally vetted each and every developer that was on every one of my client’s projects. I would do the architecture and the remote devs would do the coding.

I have stories about the various companies that I will share one of these days. But let’s just say that you really have to be very very careful when you deal with offshore development companies. The old adage of “Familiarity Breeds Contempt” is true!

John Schnipkoweit

Thanks, Marc – really great to read all your insights this morning and I appreciate the time you’ve taken to respond to everyone!

Matt Bye

Great thread. Apols if this has already been asked – how did you get into the fractional work? Thank you!

Marc Adler

Actually, nobody asked this yet, so thank you.

For many years, I had been constantly approached by people who had ideas for applications but had no idea how to build them. Invariably, they would get directed to me. Due to the time commitments that I had from my day jobs, I did not have the time to fully engage with these founders. I would often say “Let’s go to lunch or buy me a beer or three, and I will sketch out how you would hire a dev team and build your app.”

In late 2018, I reached a milestone in my life where I always said that I would retire from the workforce. When I went into “semi-retirement”, all of my friends told me that I would never be able to sit still. And they were right. So I decided to help these founders whom I had previously not been able to help, but this time, I would make it a small business, just to earn some pizza money.

I had no idea that there was such a thing as a “fractional CTO”. I just knew that I could take a non-technical founder from an idea to a full product. So CTO as a Service was born.

I started out with a single client … an EdTech company to whom I had been giving some free advice. I asked them if they knew any companies that might need my help and they said “We would love to hire you”. So I had my first client. Then word-of-mouth started to kick in.

Matt Bye

That’s great – thank you, appreciate you taking the time to reply!

Jeff Barrett

Have you run into fractional roles at other levels, say VP engineering or below? How do you differentiate between fractional CTO and being a “consultant” or “advisor”? Do you worry about executive buy-in for your engagement as much as a fractional CTO as you would as a full-time CTO?

Marc Adler

Thanks for the questions, Jeff.

Yes, there are definitely roles for fCEOs, fCPOs, and fVPEs. There are some firms that specialize in providing these kinds of talented individuals to other companies. Just the other day, on one of the Slack groups, someone was asking about hiring an fCPO (Chief Product Officer), and they got some recommendations.

> How do you differentiate between fractional CTO and being a “consultant” or “advisor”?

That’s a really interesting question. In my mind, the fCTO gets a lot more hands-on than an “advisor”. In my way of being a CTO, I can do the architectures, run the dev organizations, set up and administer the cloud environment, choose the technologies that are used, and more. To me, the “advisor” might play a lot more passive role and a role that is a few thousand feet higher (although I have certainly done that job as well). The “consultant” might be tied to a single client and may look to work 40+ hours per week.

> Do you worry about executive buy-in for your engagement as much as a fractional CTO as you would as a full-time CTO?

In my model and my preferred way of working, I usually work with non-technical founders. And I consider every facet of a technical decision, including costs and risks. So, for me, it’s usually easy for me to get the buy-in of the founder since I am the one who is usually the much-more experienced person.

Jeff Barrett

Marc Adler Thank you for your perspective here. So fCTO is closer to the company day to day and has a deep commitment to the work being provided. This makes sense.

Are you leaving artifacts of your decision processes for your ultimate successor?

Is a part of your engagement model how you will disengage with your client?

Do you set an expectation of helping them hire a full-time CTO when the timing is right?

Marc Adler

Great follow-up questions, Jeff.

> Are you leaving artifacts of your decision processes for your ultimate successor?

Yes, certainly. People who know me know that I am obsessive about documentation and memorializing all design decisions. I usually insist that all of my clients have some sort of Wiki where all technical decisions can be found. I also am heavily involved in interviewing my successors.

> Is a part of your engagement model how you will disengage with your client?

It is not written as part of the contract. But this is something that I talk to my founders about … that if and when they start getting in some revenue or funding, they should look to bring on either a full-time CTO or VPE. But the founder should never be left alone with the remote development company.

Jeff Barrett

Marc Adler is it always a remote dev shop or are you ever hiring engineers into the company? I ask because to me it would feel off to hire a team for another CTO to take over so early in a company’s life.

Marc Adler

I almost always use a remote dev company. And most of the CTOs that have followed me have retained the original dev company. One of the advantages of having an fCTO like me involved is that I don’t let the dev company get away with that many mistakes, so usually, the CTO that follows me has a well-functioning team.

Jeff Barrett

Marc Adler very interesting so are you also hiring in the dev shop or are they often already in place? If hiring in the dev shop yourself is it often the same shop?

Marc Adler

For most of my startup clients, I have to hire the dev shop, but it is up to the dev shop to furnish me with their developers. I do not like it when a dev shop goes to the market to hire developers for my project. I always prefer that the dev shop has the talent on hand … probably from their bench.

Jeff Barrett

Thanks for all of the answers Marc; I really appreciate it!

Mark Batty

Hi Marc, would have joined sooner but I am suffering with Covid 🙁

I assume you are US based, so curious about how much (if any) work you take in UK/Europe; and related to that is how is fractional CTO work different between the US and UK/Europe.

Marc Adler

Ugh, feel better Mark. I had it a few weeks ago, and luckily my case was very mild (just feeling tired for a few days). But another CTO that I was with at the time had symptoms that were much worse, so take care of yourself.

I have had a few clients overseas, most notably in England. I think that the hardest part is my overseas clients getting the wire transfers right 🙂

But, generally, the work is the same between the USA (and Canada) and the European countries. The most difficult thing that you have to navigate is the regulatory requirements in each country. Things that are not kosher to do in the USA could be perfectly legal in another country and vice-versa.

I have to sadly admit that I lost out on a prospective client last week. He was in Oxford, and despite having some great conversations with him, he decided to choose someone that was local to him. This made sense since it was a health tech company, and the person that he eventually chose was familiar with the UK health regulations.

Mark Batty

Thanks for the insight, Marc Adler.

Much appreciated; symptoms not too bad (my wife had it a week before me and was in bed for 2 days); but mine is dragging on 🙁

Support

Dominic Tancredi

Love the AMA Marc.

What’s in your utility belt, as in for strategic decks, technical audits, etc? I’m always curious about what other CTOs carry on them for analyses.

Also is there a project type you most enjoy and also find most challenging?

Marc Adler

Hiya Dom. Great to see you here.

Over the years, I have accumulated lots of artifacts that I can pull out when needed. Technical due diligence lists, case studies, and PowerPoints that I have presented to both executive boards and to investors. Plus, other CTOs contribute and freely share their own lists as well, so there is a wealth of info to choose from.

As far as things that I like to do … anything where I can do some juicy architecture work. I love designing and tinkering around with stuff. There are always interesting design decisions to be made around things like … scaling … where there are ten different solutions that can always be considered.

The most challenging is tackling a new domain that is completely foreign to me … especially when it involves math (ie: quant stuff).

On a non-technical aspect, another challenging part of the fCTO job is dealing with developers who do not speak English well and the differences between a hard-driving cigar-chomping NYC type-A personality and a foreign culture.

Kathy Keating

When you have open space in your calendar for a new client the pull to take on new work is real. What are your criteria for accepting or rejecting new work?

Marc Adler

Hello to Kathy, one of my favorite CTOs!

I freely admit that I am very bad at saying “no” to potential clients. Everyone seems to have a need, and I am flattered that they would come to me for help.

That being said, there are a few limiting factors that I have on deciding to take on new clients.

1. Time. Right now, the most precious commodity for me is free time. After almost 40 years in IT, I want to do other things in my life besides CTO-ing. So, I limit the amount of time I spend with all clients to 20 hours a week max. Therefore, I will not take on any clients that want me full-time. I made an exception with XP (where I took a full-time job), but that was a situation I could not say “no” to.

2. Social mission. I will not take on clients that have a mission that is contrary to my social values. I will not take on anything that is related to cryptocurrencies. And now, I will not take on any work that involves capital markets. However, I *will* give priority to startups that want to improve any aspect of our lives.

3. Tech. As I mentioned in a previous comment, I am tending toward rejecting all jobs that involve “rescues”. It’s difficult to mine the depths of other people’s mistakes. I would like to be involved with a client when it is totally greenfield.

4. Due Diligence. I actually enjoy technical due diligence work. I get to dive into a new domain very deeply for a few weeks, and I do not have to worry about cleaning up other people’s mistakes. I just have a responsibility to the investor that hired me. So I will give preference to DD work.

Kathy Keating

Marc Adler thanks! You’re one of my favorite CTOs as well!

I tend to be really good at rescues which is why I’ve done so many. I think in the future if I do another rescue, I’ll make it an interim situation.

I’m prior times I would do a rescue, and when the rescue is complete and back on track – I’m left at a company that doesn’t match my values (do good in the world).

I appreciate your ethos there. It’s nice to be at a point in my career where I can be more selective.

Kathy Keating

I really appreciate your sage advice, Marc! this AMA is so informative!

Mark Robinson

What have you found to be the most successful approach when selling your CaaS service to larger clients, who typically don’t engage with smaller companies/single-seat consultancies? From the posts I’ve seen, this doesn’t seem to have been too much of an issue for you.

Marc Adler

Thanks for the question, Mark. I wouldn’t say that I have been totally successful at landing larger companies as clients, but I haven’t been a failure either because I tend to stay away from larger companies. The retainer model that I have (you pay me upfront for a certain number of hours per month and you draw down from those hours) is not the billing model that most larger companies can deal with, and one of the things that I do not want to do is rack up thousands of dollars in receivables and start chasing after the money with companies who have a net-90 day policy for payment (just look at how Elon is currently stiffing his vendors). I also tend to stay away from larger companies because of the friction that is inevitable when trying to implement something quickly.

That being said, the larger companies that I engaged with (ie: XP) were the result of a personal recommendation or a personal contact.

All said, you need to ask yourself a question… would I rather work for a small single-person startup and charge $X per hour, or would I rather help a large company and charge $X*3 per hour? For me, at this stage of my career, helping small startups is more important. Otherwise, I might still be with XP 🙂

Like

Kunjie Qian

Hi Marc, how do you teach yourself new subjects? What is your approach to judgment and decision-making?

Marc Adler

Great questions, Kunjie.

Teaching myself new subjects … blogs and YouTube! I find that, as I get older, I do not have much patience to sit down with a good textbook like I used to. I subscribe to tons of blogs and depending on the subject, you can always find decent intro-level material on YouTube. As far as tech stuff goes, I still code, so I find myself firing up my favorite IDEs and writing small proofs of concept in order to dive into something that I need to learn.

As far as judgment and decision-making … I am going to talk about what I do now as opposed to what I did 20 years ago. I find that getting feedback from peers is very important, and I rely on some Slack groups that have experienced people on it. I often will bounce an idea off of the folks in a certain Slack channel, and I am not only guaranteed to get some good feedback, but I can often find people who have gone down the same route as I have.

Kunj Qian

Marc Adler Thank you, Marc!

Viral (Veer) Lalan

Couple of questions: How do you charge a client for providing fractional CTO services? Also, in your experience what length (in time- weekly, monthly, etc) of retainers work the best for this kind of service?

Marc Adler

Thanks for asking this, Viral.

In my early days of CaaS, I would charge different amounts to different clients. I would always charge on an hourly basis because one of the mantras of CaaS was that a client could use me as much or as little as they wanted to. They could use me for a few hours per month, or they could actually book up to 10 hours per week.

The sliding scale would be based on the type of company it was, and the mission that it would be involved in. I tended to give the lowest rates to self-funded founders who had a mission that was in line with my own social values. Larger companies would be charged a higher rate. There would also be a standard fee for due diligence exercises.

In CaaS, the retainers would be paid each month and the founders would draw down from that retainer, much like you do when you retain a lawyer. As the project would get more mature, the founders would typically start cutting back on the hours a bit, making my exit a very gradual one. I found that staying 6 months with a founder is a good measure of time. Sometimes, I would be asked to stick around as a board member, or I would be paid out-of-band to meet with an investor.

Viral (Veer) Lalan

Marc Adler Thanks for the answer to the questions and sharing your wisdom. Appreciate it.

John O’Sullivan

OK – I’m in! Mainly because I may be copying your playbook in London in a few years. Q1: proportion of inbound vs outbound sales? How much work from existing contacts from your professional life in trading tech, versus new contacts from events, trade shows, and word of mouth? Q2: the shape of typical engagements. Rescuing failed projects vs tech due diligence for investors vs hiring and team build-out for non-tech founders. Or startup vs established SME vs corp clients. Or hand on engineering vs strategy advice? Length of engagement, how many hours per week?

Marc Adler

Hi John. Great to have you here, and looking forward to you starting the London branch of CaaS 🙂

> How much work from existing contacts from your professional life in trading tech, versus new contacts from events, trade shows, and word of mouth?

Actually, except for XP Investments (which was a client, thanks to a Quant that I worked with at Citi many moons ago), none of my clients came from trading tech. Although my long background in trading tech helped with my bona-fides.

I never did cold outreach to prospects. As you know, CaaS was a part-time, post-semi-retirement gig for me. So, I did not want to actively grow the business. So things like events, trade shows, etc were avenues that I did not pursue. I also had lots of offers from salespeople to help me grow my business, and I refused all of them.

All of my contracts were from either word-of-mouth or from my CTO0-as-a-Service website. I used to be one of the only “CTO-as-a-Service” people out there, so it was easy to find me through a Google search. But now everyone wants to be a fractional CTO, so it’s a bit harder to get noticed through Google searches.

> shape of typical engagements. Rescuing failed projects vs tech due diligence for investors vs hiring and team build-out for non-tech founders. Or startup vs established SME vs corp clients. Or hand on engineering vs strategy advice? Length of engagement, how many hours per week?

I definitely had my share of rescuing failed projects. As I have mentioned many times, non-tech founders should never go into a venture without a senior-level tech person by their side. So, I had my fair of “I gave the job to a company from country XYZ and they delivered a piece of crap to me.” As I got more clients, I tended to deflect those kinds of jobs to other CTOs who wanted the work.

My sweet spot was starting with a founder from Day Zero. Helping them shape the product, hiring the development team (including interviewing every candidate), helping them write the product requirements, etc. That’s what I really prefer rather than coming late into the game and figuring out someone else’s work.

Marc Adler

The length of engagement would typically be 6 months. Then I would encourage the founder to “fire me” and try to find a full-time CTO if their finances would allow it. I always limited my hours to 20 per week, and the hours would usually be heavier at the start, when I am defining the architecture, choosing the dev team, setting up the cloud, etc. Then it becomes a routine of a few meetings per week with the founders and the developers, and often, with the executives of the outsourcing company.

Marc Adler

I also did a bunch of tech due diligence work. It would usually take about 3 weeks to properly evaluate a company. Not 3 weeks of full-time work … but I feel that I had to give the proper amount of thought to my work because there was money at stake and a false negative could have consequences for the acquiring firm.

John O’Sullivan

Marc Adler cheer Marc!

Claus Höfele

Thanks for doing this AMA!

My question: different companies need different types of CTOs. How do you match your skills to the companies that you work for?

Marc Adler

Hi Claus. Thanks for asking this interesting question.

I consider myself to be very much of a generalist, so I can take on most any CTO duties. However, there are domains that I might need to learn in order to successfully engage with a client, or alternatively, I try to bring on a specialist for the engagement (if my client’s budget will allow it). For example, in my CaaS work, I taught myself things about healthcare, EdTech, real estate, and more.

As far as “different types of CTOs” … I usually like to operate at a high level. I am not the kind of CTO that a company would hire to do coding (although I am very much still capable of that). You are so right in saying that there are different types of CTOs, and I pitch myself as a CTO that will operate at a 1000-meter level, although I still get my hands very dirty with architecture.

Mikael Schirru

Hello Marc! Sorry if one of the first questions is not related to the world of CTOs.

As an experienced IT professional, would you rather be a generalist or a specialist in some field (or even language) today?

Marc Adler

Hi Mikael, you can ask me anything, even if it’s not related to the world of CTOs.

I have always had good success at being a generalist and being able to teach myself new domains when the need arises. And I have been fortunate to be able to take some sabbaticals in order to learn new technology. For example, in 2017, I took a few months off to write an Uber clone because I wanted to dive into Scala and Akka.

That being said, there is a lot of demand out there for low-level C++ coders, especially in high-frequency trading companies. There is/was also demand for specialist languages like Q (used for the KDB+ database). In my opinion, if you are passionate about any special domain, then it’s good to do a really deep dive into that domain. But you also have to make sure that you are not “typecast” into a certain specialty (ie: “Oh, that Mikael … he only knows C++ and trading systems, and we cannot consider him for anything else.”)

Oswaldo Fratini Filho, MSc

I thought to talk with you about Marimba… 🙂

My father is a luthier and makes a lot of Cuíca nowadays! I will take a picture and send it to you here.

I am a specialist in C++ / Qt and have so good experience in hardware production/maintenance today I have been working with BLE (Bluetooth Low Energy). I am studying radio and all software stack.

The dollar is very expensive in Brazil and electronic components are missing in the world.

Do you see with good eyes a company that makes reverse engineering, recycles electronic components, and offers new electronic products with innovative software to its clients in this scenario?

Do you understand that this global and economic scenario in Brazil should remain?

I think about being a CTO of a startup to make this.

Marc Adler

Hi Oswaldo. I have been studying classical guitar for one month, and I would love to see your father’s work!

Your idea sounds very worthwhile. Of course, I know how expensive software and electronics are in Brazil. If you are able to solve all of the issues with Brazil’s legal and regulatory system, I think that you can think about pitching your idea and getting some startup funding. Is there an organization like TechStars in Brazil? There should be organizations that will assist local startups.

You should check out the YCombinator Startup School for some ideas about how to proceed.

https://www.startupschool.org/

The other day, I was having a discussion with the CTO of another company about the issues that they were experiencing when they had spikes in traffic. They had tried solutions involving CQRS, queueing, etc, but they were still having issues with the spikes in volume. I followed up with the CTO and she said she had managed to solve the issue by tuning various database queries.

It reminded me of the time when I was the Chief Architect of a “megacorp”. At that time, I controlled the consulting dollars for Microsoft. My predecessors would happily accept those hours and the bills that Microsoft gave them, but I examined every hour with a fine-toothed comb. At year-end, I managed to have a surplus of consulting hours which I needed to use. I brought in a team of consultants to examine some of our slowest apps.

It seems that there were a lot of issues related to the way that our databases were being used. Extremely inefficient queries. Non-existent indexes. These consultants went to work, and managed to reduce a very painful operation from 70 seconds to 3 seconds!

When you are doing exercises in improving performance, don’t ignore the most basic part of a system. There can be gold in bringing in a specialist to look at the queries which have been running your systems for years.

2020!!! What a year it was!!!! I hope there will never be another year like this one!

That being said …. it was actually a great year for CTO as a Service.

A Bit of Background

First, a bit of a recap about CTO as a Service for those encountering me for the first time. I spent 30 years in the world of computers, as a developer, the owner of a software company, a consultant, and for the past 15 years, Chief Architect or CTO to some very large companies and some very small companies. In November 2018, having reached several milestones that were set out early in my career, I decided to leave the workforce.

For years, people had asked me to listen to their ideas and help them create apps around these ideas. I was too busy with my regular job responsibilities to take on a part-time gig, but I told these people that they could buy me a few drinks and I would tell them how to go about building the app. When I left the workforce, I decided to “hang a shingle” outside my door and advertise myself as “CTO as a Service”. I would spend a few hours a week with each client, and I would do everything from soup-to-nuts (except writing the code, because my clients could get good coding talent from an offshore dev shop at a fraction of what they pay me). I would help them write the business requirements, hire the development team, write the technical architecture, set up the cloud infrastructure, and manage the development team.

My goal was to spend no more than 20 hours per week helping startups. I hoped to be able to make a bit of beer money and to occupy my mind with some interesting problems.

Utah or Bust

2019 was my first full year and I was fortunate to have a few good clients to keep me busy. I was able to take the month of February off and go to New Zealand and Australia and hike the 5-day Milford Track trail.

In September 2019, my wife and I decided to try the snowbird thing for the first time, and we booked a home through HomeAway in St George, Utah. The plan was that we were going to stay there for 2 months. I would work in the morning and spend the afternoon hiking. Of course, nobody expected a little news item to hit the airwaves as soon as I got out to Utah. That little piece of news was about some virus that was running around China.

St. George is tucked in the southwest corner of the state, and we were at least 150 miles from any population center (Las Vegas, Nv). I didn’t worry much about the pandemic, and I carried on with my plan. Mornings were spent with a few clients, which included a real-estate company that used VR to show apartments, an investment platform for people who want to invest in ESG-related stocks, an FX Hedging platform, and a few other minor ones. I was also starting with a new client who was developing a platform for meditation and wellness.

On February 15th, everything was locked-down in NYC. My daughter’s medical school rotations were canceled, and she took a flight the next day to St. George. She booked a 1-week ticket, hoping that things would return to normal soon. She ended up spending 2-½ months with us in Utah!

Petrified Dunes, Snow Canyon State Park, Ivins, Utah

The Startup Craze

I thought that my business would go under. Who would want to embark on a new startup during a pandemic? Well, I was wrong. In the 2 weeks following the start of the lockdown, I signed 4 new clients. 

1. An InsureTech startup whose mission was to put sensors in various businesses and warn the owners about abnormal conditions in their business. They would work with various Property and Casualty providers in the insurance space to promote the use of this iOS platform with the providers’ customers. This was an interesting project because it exposed me to the world of Samsung SmartThings sensors. The platform was based on AWS Lambda, DynamoDB, and Kinesis.
2. A wellness startup founded by a gynecologist that wanted to educate high school students about sex education. They came to me with a little bit of code that was written by a consultant. I hired the development team for the founders, ran a few meetings, and then stepped back. The gynecologist and his partner decided that they could be project managers!
3. Another wellness startup that matched couples who were having pregnancy-related issues with health providers. Another project where I did everything from soup-to-nuts. I wrote the BRD, designed the architecture, hired the development team, and ran the project. 
4. A platform that lets businesses register “phrases” and websites associated with those phrases. When someone talks into their mobile device, they can phrase a question (i.e., show me the best Japanese restaurant in NYC) and the platform would see if that phrase matched any of their licensed phrases, and if so, brings up the associated website.
5. I also did a few tech assessments for some private equity firms.

I could not explain the sudden surge in business. My guess is that the impending pandemic motivated people who had ideas to finally take their shot at realizing their dreams. Kind of a “now or never” attitude.

Meanwhile, our host in Utah, seeing how the pandemic was hitting NYC, graciously offered to extend our stay indefinitely. We ended up spending an amazing additional two months out there, waking up every morning to red-colored mountains staring us in the face.

Backle to The Apple

At the end of May, we packed the car and spent 5 days driving back to NYC. The drive eastward out of Taos, New Mexico was one of the most picturesque drives I have ever made. The flat openness of eastern New Mexico is really something to behold. An unexpected pleasure was driving through West Virginia, where there is no such thing as a level road.

Outside of Taos, NM

We got back to NYC on the first night of the curfew (due to the riots and looting associated with the killing of George Floyd, amongst others). I was the only car driving through the Holland Tunnel in the middle of rush hour, and it was an eerie feeling. I could never envision NYC as a ghost town, but there it was. It was certainly quite a shock after being relatively isolated in Utah for the past 4 months.

Continuing On

I started with yet another new client, a Canadian architecture firm that wanted to modernize their old ASP.NET/PHP app. I hired my go-to development firm, and we dug right into porting the platform to ASP.NET Core and EntityFramework Core.

Soon after, an old colleague called me to tell me that the Principal Engineer of his company had left and that they needed someone to fill in for a while. This company was not a startup … they are a mid-sized company devoted to HR and Payroll software.

An old client from England got in touch with me to say that he had secured some funding and asked me to help him. His mission was to do financial planning and budget for people in the lower-income bracket in England. A very worthy mission. The interesting aspect of this app is that it is totally driven by WhatsApp.

Over the summer, I started with a new client whose mission was to gameify the employee education experience. We built a VR world in which employees walk through, clicking on objects within a virtual office and “playing” knowledge-based game. This was a microservice-based architecture, with ASP.NET Core on the backend, MongoDB as the persistence layer, and React/Unity on the frontend. I did the usual tasks … BRD, hiring and running the dev team.

I got another client in a very interesting way. That client was using an offshore dev shop. This client got into a dispute with the dev shop over the hours that were billed, and the dev shop made a copy of their Github repo and deleted the client’s repo. The client did not know how to get their code back, and they were in a bad predicament. I managed to recover their code and helped them hire a new dev shop. Meanwhile, I did a tech assessment of their platform and the code that the old dev shop wrote and came up with a roadmap for future development.

I was hired for a few more tech assessments by a VC firm. The great thing about these assessments is that I am often exposed to new domains and new technologies. 2 of the assessment were for firms in the InsureTech space, and the other was for a company that provided parking technology. I got to learn a bit about LPR (License Plate Recognition) technology and the associated hardware. I also got a chance to re-learn a bit of Scala and Akka

I ended the year with a brand new client who is building an app for the real-estate space, with the idea of making complex real estate transactions more friendly. Anyone who has gone through the process of buying or selling a home should be interested in this app. Another soup-to-nuts project for me. I even got a chance to write some Typescript/NodeJS code.

I start 2021 with a new client from Toronto who is building an app in the vacation/experience space. I am very excited to be taking them on this journey.

Final Thoughts

In summary … I had 15 different clients in 2020. If I extrapolate my revenue to a 40 hour week, then it was quite a good year on a financial basis. I only wish that we could leave our apartment in NYC so that I could enjoy it!

One of the great things about being an interim CTO is the fact that I get exposed to so many new domains and technologies. 2020 featured technologies having to do with VR, IoT, image recognition, and more. InsureTech is hot, as is the very-needed area of wellness.

Not everything is a bowl of gravy. One thing that is certain to make the voyage rougher is when a non-technical founder, in order to save costs, decides to take on the project and dev-team management duties themselves. There are many aspects to managing a dev team that have to be learned over a long period of time. Developers, especially off-shore developers, get extremely frustrated when they don’t see the proper leadership at the helm, much like musicians would rebel if I got up in front of the New York Philharmonic and started conducting the orchestra.

I am supremely grateful that The Fates smiled down on me and gave me a prosperous year amongst all of the sorrow that was happening in the world. Let’s hope for a much more sane and safe 2021.

Marc Adler

CTO as a Service

February 1, 2021

In my CTO as a Service consultancy, I tend to use various remote development companies to write the code for my clients. Even though I still code after all of these years, I am usually not cost-effective for my founders in a coding role. (But I still do the architecture, all of the set up on the cloud platform, and occasionally write some critical code.)

When the engagement with the development team ends, there is a wrap-up process that takes a few days to accomplish. If you have a remote development team, then you should allow them at least 3 or 4 days to perform these final items.

I would love to hear about your own list that you go through in order to wrap up an engagement.

(Note: Even though I mention AWS in this article, you can substitute any cloud provider, such as Azure or GCP.)

---

Tasks for Wrapping Up

Documentation

• Read, edit, and check the documentation that the consultants write so that it can be handed off to another development team
• We need the following documentation
• Architecture diagrams, including integrations with 3d-party services
• Structure and flow of the backend and frontend code, including interaction with 3rd party services
• Deployment instructions for all environments
• Troubleshooting guide

Code

• Code review (if time allows)
• Go over the Jira to see what was not done and to see if anything was left in a partial state
• Make sure that all tests are passing
• Making sure that all code has been merged to Main
• Make sure that we have access to all secret keys and PEM files
• Make sure that we have access to all licensing and credentials for 3rd party libraries and systems
• Doing a deployment on both test and production to make sure that we know how to do it ourselves

AWS

• Checking out the production database to make sure that everything is smooth
• Make sure that all routes are correct in Route53
• Tighten permissions on S3 buckets
• Closing down all open permissions on the various machines
• Possibly removing the consulting team from Github, AWS, 1Password, Twilio, Stripe, etc

Monitoring

• Install Monit on all machines
• Setting up various CloudWatch alarms on AWS

---

CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. I work with all sizes of companies, but I love helping startups realize their visions. Services include hiring and leading development teams, architecture, code and architecture reviews, roadmaps, coding, speaking with investors, etc.

Looking back on my first full-year of CTO as a Service…. some statistics.

I had 13 clients. 6 are currently active.

Out of the other 7 clients, I helped 3 of them hire their permanent CTO or VPE, and I transitioned out of those companies. 2 were short-term advisory or architecture review gigs. 2 companies called it quits because the founders ran out of funding.

The clients were from EdTech, HealthTech, SelfHelp services, RealEstate, LegalTech, large investment banking, FoodTech, social media, investments and trading platforms, and financial help services.

Client 14, an InsurTech startup, just signed with me this week. I am most excited to get exposure to AWS IoT Core.

My goal was to keep myself busy 20 hours per week in my “semi-retirement”. I easily met this goal.

All of my clients were on AWS. Not a single client was on Azure or GCP.

Languages and platforms used were mainly C#/ASP.NET Core and Typescript/NodeJS.

Databases were split between MySQL, SQL Server, Postgres, and Mongo.

JIRA, Confluence, and Slack were other tools that were heavily used.

I am really looking forward to 2020. All in all, I am happy with my decision to leave the BigCo environment, and devote my time to helping small companies get to the next stage.

#cto #startupcommunity #ctosummit #startups

Marc Adler

CTO as a Service

On almost a daily basis, I receive email from non-technical startup founders who are looking for a CTO. A good percentage of these founders have little or no money to devote to their startup, but they have a dream, which is always the first step.

The solicitation from these founders usually starts off with “I am looking for a technical co-founder, but I can only pay with equity for now.” I wish that I could devote a slice of my time to assist each and every one of these founders, but in reality, I do not work for equity anymore. I would gladly give my technical advice for no charge for certain social causes that I believe in, but my current business model is to charge an hourly rate or a monthly retainer.

Do they need a CTO right now? Do they need a technical co-founder at this point? This is one of the most popular questions that I get at my monthly Startup Lean Coffee meetup.

First, the startup founder needs to consider what they are going to build as a first step. Are they looking to create a Minimum Viable Product (MVP)? Or are they looking to develop the complete working product? The MVP can have just enough functionality in it to prove out the idea and to attract further investment. Depending on the answer to this question, they may want to bring in an experienced technologist for just a few hours to sketch out what the technical architecture will look like.

Most of the time, they just need someone to code the MVP. If they have enough budget, then they could use a CTO-type advisor like myself to guide them through the technical aspects of creating their application and pitching the MVP to investors. But, often time, they need some coding help that can be paid for in equity rather than in cash.

If you are a startup founder with a limited budget, and you need to develop your MVP, don’t despair. There are a number of routes that you can take to develop a basic MVP that is designed to attract more investment.

Strategies for Creating the MVP

Low-Code/No-Code

In order to test out the viability of your idea, you can create a “demonstrator” of your idea. This demonstrator can be as simple as a few web pages stitched together, using fake test data that is stored in a few files on your computer.

No-code or low-code development environments have been around for many years. These platforms let you drag-and-drop visual elements onto a canvas, and create a fully-functioning webpage. You can create a number of pages and hook them together through various kinds of user interactions, such as clicking on a link or a button.

The nice thing about these platforms is that you do not have to code anything yourself (or, perhaps, very little coding may be needed if you need more complex functionality). Some of the platforms will let you automatically integrate with various third-party services. You may want to hire a graphics designer to create some custom artwork that you can incorporate into your demonstrator. You may want to consider hiring a User Experience (UX) specialist to make sure that your application flows together in a logical and pleasing way. But you will not have to spend any money on hiring a developer.

Try to code up a demonstrator version of your product using a web-based design platform such as Balsamiq, Sketch, Figma, or InVision. A more advanced low-code platform whose popularity is increasing is Bubble. There are integration platforms such as Zapier that can be used to hook up various third-party services.

Teach Yourself How to Code

There is nothing like getting into the weeds and coding up what you want. Many people have taken a programming course or two in school and might be a bit rusty. If you have not learned how to code, there are courses that you can take at local community colleges, night schools, and online. For example, Udemy has a course in Introduction to Programming. Other MOOCs like Udacity and edX have additional courses. If you want to self-pace yourself and get a wide variety of content, I like Pluralsight.

The courses mentioned above are fairly inexpensive. Once you learn to code, you don’t have to pay another programmer, unless you need some sort of special skill.

Approach a Recent Bootcamp Graduate

There are many coding bootcamps that new programmers go to in order to get intensive training in coding skills. These bootcamps, like Flatiron School or General Assembly, often last for several months and teach the new coder a wide variety of technical skills. The bootcamps will try to find their new graduates a programming job, but these graduates often face the chicken-and-egg problem – many companies want to hire developers that have some amount of experience, but how does one get that experience?

These graduates are hungry for some experience that they can put on their resume. They might be receptive to an equity-only arrangement for a few months while coding up your MVP. There is nothing like having the satisfaction of bringing up a website and saying “I wrote this”.

My only word of caution is that these graduates learned a lot of new technologies in a short period of time, without much practical experience in applying what they learned. Be wary of “resume-oriented development” where the new graduate will want to use a certain technology because they want it on their resume. An architecture and/or code review by an experienced CTO will help identify inappropriate technologies that might come back to bite you later.

Use an Offshore Contractor

You can usually find an independent coder in countries like Russia, India, and Vietnam who charge a fraction of what you would pay a developer in the USA or England. You can find good developers for as little as $20 per hour, or you can negotiate a fee that is outcome-based. 

How do you find these developers? There is the ever-popular CraigsList, which I personally would never use, as you are subject to being contacted by lots of scammers. You can contact a local college and post something on their job board. Recently, a number of platforms have emerged where freelance developers advertise their availability while you advertise your project.

A popular platform for finding these types of coders is Upwork. You post your project on Upwork, and you get “proposals” from developers who are interested in working on your project. After examining all of the proposals and interviewing the various developers, you pick the developer who you would like to work with. Developers who have done projects before through Upwork are rated by their customers (and the developers also rate their clients).

One word of advice – Upwork takes a certain percentage of your payment to the developer. This percentage decreases as the developer bills more hours to your project. Keep in mind that the developer will not receive 100% of your fee, so you may want to adjust your fee so that the coder gets a fair wage.

Incubators and Accelerators

There are various incubators, accelerators, and Angel Investment funds that will fund you if they like your pitch. In New York City alone, there are probably a few hundred of these types of places that you can go for some initial funding. There are giant incubators and accelerators, like Techstars and Y Combinator. Many startups apply to these incubators, and the acceptance rate is very low. There are more boutique accelerators like Brooklyn Bridge Ventures. If you are a woman-owned startup, and especially a minority woman-owned startup, then Pipeline Angels is a great place to consider. And some universities, like NYU, have their own accelerators that are designed to fund their graduates.

I help run the Startup Lean Coffee monthly meetup at Betaworks Studios in New York. Betaworks is a space where entrepreneurs gather and exchange ideas about their startups. They also host almost nightly meetings for the startup community.

Some of these accelerators will want to see a senior-level tech person that you are associated with before they will give you funding. Techstars is well-known for this requirement. If you need to have that kind of CTO-like person involved with you when meeting investors, then this is a service that CTO as a Service can provide.

Grants

There are a number of grants available at the state or local level for certain startups. These grants usually come with no strings attached. You do not have to give away any equity in exchange for these grants. Of course, there is a lot of competition for these kinds of grants, and they are usually awarded to startups that are working on ideas that will improve society in some way.

Final Words

If you are a startup founder who has a great idea, then congratulations …. you have taken your first big step. Even if you do not have any money right now to take your idea to the next step, there are several different avenues for you to pursue in order to develop your MVP for little or no money. Some of these avenues will require you to spend a little bit of money, and you may be able to get that money through your savings or through Friends-and-Family investments. Some of these avenues will require you to give up some equity.

No matter which route you decide to take, make sure that you always have a senior technical person who is watching out for your interests by your side during the early stages.

---

CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. Please consider CTO as a Service to be your senior technical advisor on any projects that you might develop.

Marc Adler

CTO as a Service

November 2019

---

Thanks to Kathy Keating, George Sudarkoff, Dave O’Flynn, Glenn Proctor, and Adrian Howard for their suggestions.

Marc Adler

CTO as a Service

(Note: This article is written for the non-technical founder of a potential startup. Nevertheless, the questions listed below can be useful for even the most experienced IT executive who is in search of a new remote development shop. In addition, the article assumes that the founder is based in the USA.)

So, you have decided to break free of corporate life and pursue that dream of turning your big idea into a startup company.

Congratulations !!!!!

In all likelihood, you are not a technical founder. You may have taken a few programming classes in college, or maybe you even spent a few thousand dollars and went to one of those Coding Bootcamps that teach you how to code in 8 weeks. But, in reality, you probably don’t trust yourself to sit there and write an app that will be the lifeblood of your startup.

You have probably squirreled away a bit of money from each paycheck in order to live your dream one day. You may have even gotten some friends and family to chip in a few dollars. You might quit your day job to devote full time to your new startup, or perhaps you will still work that job while waiting for the app to be developed.

So, what are your next steps?

Hopefully, you will have “rented” a CTO for a while … someone like me … to guide you through your next steps. After all, you will need someone very technical who represents your interests so that you will not end up with nasty smells in your shiny new app. 

(Read this article about code smells and remote development teams.)

---

Building Your MVP

Even if you don’t have an interim CTO, you still need someone to develop your app. Most startup founders will develop an MVP, which is short for Minimum Viable Product. This is a lightweight version of your full app which is designed to provide the necessary functionality while whetting the appetite of potential investors.

You hopefully have at least $50,000 in funds which will be used to develop the MVP. That amount is the finger-in-the-air estimate that I always give to my clients. You probably want a web-based version of your app, as well as a mobile version (iOS and, possibly, Android).

The more that you can do yourself, the more you will be able to devote to the development costs. For example, you might try to write up the business and functional requirements yourself. Instead of sketching out your wireframes using pencil and paper, you might want to use an online design tool like Sketch, Figma or Balsamiq to draw out the entire user experience.

You have your idea for an app, you have some idea of how the app might look, you have your business and functional requirements written down, you might have a technical architecture, and you have some money. The next step is to get some people to actually code up the app and set up the infrastructure that the app will run on.

What Kind Of Developers Do You Need?

The most important developer that you need is the backend or server-side developer. This is the developer who will be interacting with the cloud infrastructure (assuming that you are not going to buy your own servers and that you will use a cloud provider like Amazon Web Services, Google Cloud Platform, or Microsoft Azure). This developer will develop the data models, the corresponding database schema, and will interact with a database platform like Postgres, MySql, SQL Server, or MongoDB. The backend developer will write all of the business logic of the application, will develop REST-based APIs (so that other developers can interact with your application), will handle authorization, authentication, logging, and monitoring.

As you can tell, the backend developer is the real workhorse. Their knowledge has to span multiple technologies and different layers. They will hopefully have direction from your interim CTO, who is hopefully a very technical CTO that has architected your system. The backend developer may present some key person risk, because if that developer leaves your project, then it may take several weeks to train a new backend developer on the codebase.

If you have enough funding, it is desirable to have two backend developers who can divide the sever-side tasks equally among themselves. This also will alleviate some of the key person risks, but only if the two developers do not work in complete isolation. 

The other half of the equation is filled by the frontend or User Interface developer. This is the developer who writes the visual interface of your app. They have to know technologies such as HTML, stylesheets, and they may develop with technologies like AngularJS and React. They have to write REST API calls to fetch information from the app’s backend and present the data to the user in a pleasing way. The frontend developer might be called on to develop the User Experience (often called the UX), which is the overall flow of how a user interacts with your application.

Ideally, the frontend developer can write code for both a browser-based user interface and a mobile interface. You want to find a frontend developer who has the cross-platform experience, and who knows the ins and outs of iOS and Android. They might know technologies like Xamarin and React Native.

There are other players in the development process that you might end up needing.

First, there is the QA Lead. You want someone really going through your entire app, workflow by workflow, before you release the code to the user community. Developers can try to write automated tests, but these tests will only provide a little bit of coverage. You, as the founder, can do the testing yourself, but an experienced QA Lead will know all of the hotspots to explore in an app.

Second, you may need a graphics designer. You cannot expect your frontend developer to be a graphic artist as well. Who is going to design all of those icons, who is going to come up with a good color scheme, who is going to draw all of those custom images? A good development shop might provide you with a graphics designer that will be on call.

Third, you may need a local project manager. You hope that the two developers (and maybe the QA Lead) can manage themselves. You might even want to manage the project yourself at the beginning. And your rented CTO might do the same, although he might be overpaid for that job. As the founder of a startup, your main job is to build your company and raise additional funds, so project management might not be worth your time.

Fourth, if your application is going to be doing any kind of recommendations, or you are thinking about Machine Learning in order to improve those recommendations, you may want to hire a data scientist.

Lastly, a development company might try to upsell you the services of their own CTO, CIO, or Chief Architect to do code reviews of their own developers. This might be good, but you should always have someone on your side to do this, someone who is vested in the success of your startup.

The Money Part

Let’s assume that you have $50,000 that you are devoting to the remote development shop to develop the MVP of your app. How will you spend that money?

Let’s do a bit of math.

My finger-in-the-air estimate is that a proper MVP will take 3 to 4 months to develop. This means that you have around $12,000 to $15,000 to spend each month, which translates to about $3000 per week. If you divide that $3000 into two developers, this means that you are spending around $1500 per week for each developer, or $300 per day. This is about $35 to $45 per hour for a developer.

Where are you going to get a really good developer for $35 an hour?

Most USA-based development shops will charge at least $75 per hour. Many of the really good shops that I have worked with will not take a client unless that client spends at least $30,000 a month with that shop. With your current budget, this means that you will only be able to use that development shop for about a month and a half.

You can go to an online job platform such as Upwork, which is a platform for individual developers to connect with companies that need development services. These developers usually work remotely, and you would have to coordinate all of the various individual developers that you obtain from Upwork. You would also need to be able to properly vet these developers yourself.

You can try to hire someone locally, but in all likelihood, you might be getting a student or someone with a not-so-great track record. Even recent college grads can make way more than $35 per hour. To be honest, I might be a bit suspicious if someone local was to charge $35 per hour.

Your final choice is to use a remote development shop, one that is probably in a country like Russia, Ukraine, Belarus, Poland, Romania, Colombia, Costa Rica, and Vietnam. These development shops will be fully-stocked with the talent that you need, at the price that you need. You can easily find a good senior-level developer for about $5000 per month.

Now that you have decided to explore the possibility of hiring a remote development shop, how will you choose among the thousands of them out there, all with seemingly the same kind of talent? How will you, as a non-technical founder, be able to properly choose among all of the companies who are answering your call for help?

Hopefully, your rented CTO will help a lot and will ask the hard questions to each of the shops. But, if you want to do it yourself, you have to be prepared with a list of questions that you want to ask each of the shops.

Interview Questions for the Remote Dev Shop

And now we come to the gist of this article. Below are the kinds of questions that I will ask the various remote development companies in order to help my client build an app.

Price

• What are your costs for staff?
  • by skillset (developer, QA, UX/UI, Project Manager)
  • by skill level (senior, junior, mid-level)
• Will you work on an outcome basis, or are you strictly hourly?

Of course, the price is the main make-or-break decision here. Startups have a limited amount of funds. 

We cannot hire a development team that is outside of our budget. No wiggle room here. We need the best talent for the least amount of money.

If the price is right, we can move on to the rest of our questions. If not, then we might come back to you for another project when we have more money.

Location

• Where is your company based?
• Does a development team use developers that are located in different offices? Or will a development team always be located in the same office?
• Will your developers work USA hours? What is the overlap with the East Coast/West Coast of the USA?
• Can your developers attend online dev team meetings a few times per week?
• How fluent in English are your developers? How do you manage the issue that we get a great developer who is not fluent in English?

The choice of location is important for two main reasons: collaboration and cost. 

You need to decide if you are going to want to be in direct contact with the developers of your application, and whether the outsourcing company will even allow that kind of access. If you have a small development team, you (or your CTO) are going to want to have regular development meetings with the team. The team will feel more connected with your product and may do a better job if they were to see and hear the person who is actually the brains behind the product. You can use the development meetings to clear up any issues and to clarify what you expect in case there are any misunderstandings.

If you are located in California, and you hire a team in India, then you will hardly overlap with the working hours of the developers and you may find it difficult to schedule regular team meetings. However, if you are on the West Coast and get a team that is based in Latin America, then it will be much easier.

You might be able to benefit from a country whose currency is weak against your own currency. You want to make sure that you stay around that $35-$45/hour figure, and you might be able to strike a better deal if you can pay a company in their local currency, even after the currency conversion costs. Even if the development shop quotes their prices in US dollars, you might be able to negotiate a better price if the US dollar is strong against the other currency.

Staffing

• Are the developers hired as full-time employees of your company, or do you hire on an as-needed basis?
  • If you hire on an as-needed basis, do you use local talent, or do you go to platforms such as Upwork?
• How many developers do you have, and what is the (technological) composition of your staff?
• How much turnover do you have?
  • If a developer is in the middle of a project for a client, and that developer leaves, how do you determine how to backfill?
  • If we hire a mid-level developer, and they leave, and the only person on your bench is a senior developer, would you backfill with the senior developer and charge the client the rate that they were paying for a mid-level developer?
  • If a developer leaves, how much downtime could the project expect?
  • What does your bench look like?
• How do you hire your developers? What does the screening process look like?
  • How do you determine the various levels of developers? (Senior, mid, junior)
• Is a developer assigned to multiple clients or to just one?

I want my clients to have the perception that their remote development team is no different than having an onsite team of full-time employees. This means that the team is stable, is stocked with the skill set that we need, experiences very little turnover and that all parts of the team are working as one cohesive unit.

Ideally, I want the same developers to stay with the project during the full lifecycle. I expect a developer to work a full day on my project, and I don’t want that developer to multitask between different clients. I want the developer to be totally dedicated to what I am building.

I like to use remote development companies that have relatively low turnover. In the case of turnover, I want to see the departing developer do knowledge transfer, and for the new developer to take over in a seamless fashion. To get a sense of what it is like to work for the remote development company, I might check websites like Glassdoor in order to see what the developers are saying about their company.

Skillset

• Do you have UX experts on your staff? 
  • Could we see examples of UXs that they have designed?
• How do your developers keep up to date with the latest technology?
• How do you determine what new skillsets to invest in and support?
• How do you build and maintain centers of domain expertise?

At the beginning of this article, I talked about the various roles that I may need for a project. I want to make sure that the remote development shop has a wide variety of skills available on an a la carte basis, even if I don’t need those skills right now.

Most importantly, I want to make sure that the application that I build is not only visually pleasing but has a workflow that makes logical sense So, I might ask to bring on a UX or UI expert for a little while during the project.

Company Reputation

• What makes your development company different than the others?
• If I were to go to a review site like Clutch, what would I expect people to say about your company?
• If you deliver a product which is buggy, what are your policies around make-goods?
• How many customers have you had? How many do you currently have?
• Can we get references from your past customers?
  • Can we talk to former customers?
• We would like to retain all IP rights for what you have developed for us. Do you have any issues with that?
• We would like total secrecy surrounding our startup. Will you agree to not tell your potential customers about us?

There is nothing like hiring a company that has delivered successful products with a minimum of fuss. To that end, try to find out as much about that company as possible. Look on sites like Clutch and Glassdoor for comments about the company. Try to join some startup-oriented Slack channels and see what others have to say. If the company has a USA-based sales office, try checking with their local Better Business Bureau.

Will the company take pride in their work? If so, they might offer some type of guarantee on the quality of their work. You may want to see if they will fix bugs for no cost for a certain period after the delivery of the MVP.

The Software Development Process

• Do your developers have experience in developing an application like the one we are giving you?
• Can we interview each developer before they start?
• Are there any tools that we need to provide your developers?
  • Do you provide your own licenses for your internal development needs?
• How much technical input do you need from your client in order to start developing?
• Will your client have direct access to the developers?
  • If not, then does your client have to pay for a Project Manager?
  • What is your preferred method of collaboration?
  • Does a client absolutely need a project manager from your company? Can we manage the project by ourselves?
• How do you ensure that a developer does not create key-person risk and use a technology that is relatively little-known?
• Do you do code and architecture reviews of your developer’s projects to make sure that they are maintaining sufficient quality?
• If the client has a CTO or Chief Architect, can they do the code reviews? And what is your policy if they find major problems?
• How do your developers document their work?
• Do we receive the source code once you are done?
• We would like the developers to use our Github repo. Is there any problem with that?
• What development practices do your developers like to use?
  • What development methodology do your developers prefer? (Scrum, Kanban, etc)
• How do your developers fit into the software lifecycle (do they push all the way to live, or does that come back to our company)?
• Who is responsible for infrastructure (design, changes, maintenance)?

Now that you have determined that the cost is affordable, the location lines up with your time zone, the development shop is reputable and can supply the necessary skillset that you need to develop your application, you now need to determine how you will be interacting with them on a day to day basis.

It helps to have a development team that is familiar with the domain that your app will cover. For example, if you want a stock-trading application to be written, it would be beneficial to have developers who are familiar with the domain (ie: what is a stock, what is a trade, what is an order, connectivity to an exchange or to a clearinghouse), and with the technologies that will be used to implement the application (ie: MongoDB, real-time messaging, WebSockets, etc). If the remote development shop does not have developers available with the necessary knowledge, how long will it take the development shop to recruit that talent? If training is required, are you responsible for the training, or is that an expense that the development shop will bear.

Will you have the opportunity to interview a pool of developers (or, at least, look at their resumes) for your team? Or will the development shop give you the team? Sometimes it might be best to let the dev shop handle the choosing of the team because there might be existing synergies in place. It might be a group of people who have already worked together to build successful products. At the very least, I would want to look at the resumes of each of the developers.

Software licensing costs can be huge. Because most applications are developed on the cloud using PaaS (Platform as a Service) and SaaS (Software as a Service), your licensing costs for commercial software can be part of the cloud vendor’s charge. But what if you had a specialty piece of software that you need to install on your cloud-based server? Do you need to purchase a license for every developer?

You should have direct contact with the developers on your team. Be wary of any development shops that insist that your only interface with the developers is through a paid project manager. You also want to have weekly meetings with the entire development team, using a collaboration method like Skype, Google Hangouts, or Zoom. You may want to invite the developers to a special Slack channel that can be used for real-time collaboration.

The source code should be available at all times through a cloud-hosted source control repository, such as GitHub, GitLab, or BitBucket. All code should be well documented in English, and the code should be unit tested.

Some remote development companies will offer the free services of a CIO or senior-level Architect to ensure that the code and architecture are of high quality. Even though some startups consider their MVPs to be throwaway efforts, I like to preserve as much code as possible.

---

The list of questions that this article contains is only a small portion of the questions that a trained senior technologist will look for when evaluating a remote development company. CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. Please consider CTO as a Service to be your senior technical advisor on any projects that you might develop.

Marc Adler

CTO as a Service

September 2019

---

Thanks to Andrew Cherry for reading an early version of this document.

Executive Summary

There are two main use cases for considering messaging in most applications – lightweight eventing and messaging between components, and processing large streams of data. If you are using AWS as your main platform, it makes sense to consider using AWS-native platforms to handle the messaging. For the case of simple peer-to-peer messaging, where the exact order of the messages is not mandated and a single application has to be notified, SQS is a good solution. It is easy to set up and manage, and it can be used to perform a simple event notification or be used in a request/response paradigm. For the case where a number of applications have to be notified of an event, SNS is a good solution, as it supports the concept of topics. SNS can be used to deliver notifications to a wide variety of notification mechanisms (email, SMS, Lamba, REST), and can interface with SQS queues for notifications to individual applications. If your application has a use-case where it needs a heavy amount of streaming, then Kinesis is an easy-to-use streaming platform for processing large streams of data. Kinesis Analytics can be used to perform Complex Event Processing, and to find interesting events within the streaming data. For what most applications need, Kinesis is preferable to Apache Kafka because it is easy to set up and administer, and you will avoid many of the operational complexities that usually plagues a deployment of Kafka.

---

SQS (Simple Queuing Service)

SQS provides the most basic way to perform interprocess communication – a simple queue. With SQS, you simply send a piece of data (a Message) into the tail end of the queue. One or more consumers will read messages from the front of the queue. If there are multiple consumers reading from the same queue, then any one of these consumers can receive the message that is at the front of the queue. The message is delivered once and only once to a single consumer.

SQS comes with two different kinds of queue. The Standard queue imposes no ordering of messages. The Fifo queue, which is only available in certain AWS regions, will guarantee ordering of the messages within the queue.

Polling

There are two different was that an SQS consumer can poll for messages. The first way is short polling. In this scenario, a consumer will look at the queue, read any messages that are in the queue, and then will return. If there are no messages in the queue, then the short-polling consumer will return immediately. The other way is to do long polling. In this scenario, the consumer will wait for a number of seconds for messages to appear in the queue.

Since AWS charges you for each SQS request, it may be more economical for your consumer to do long polling, since there will be fewer requests if messages are put into the queue on an infrequent basis.

Messages

A Message contains a payload, which can be any amount of data up to 256K bytes. A message can also contain custom attributes, which are name/value pairs. When a message is placed into a queue, it can have a non-zero Time-to-Live (TTL). If the message has sat in the queue without being consumed, and its TTL has expired, then the message is automatically deleted from the queue. 

If you want to have messages up to 2GB in size, AWS has a Java-based library that uses S3 as the message storage.

Scaling

You can have Cloudwatch monitor certain metrics of a queue and automatically scale out by adding additional instances.

aws autoscaling put-scaling-policy –policy-name my-sqs-scaleout-policy -–auto-scaling-group-name my-asg –scaling-adjustment 1 –adjustment-type ChangeInCapacity

JMS

The JMS way of programming is available for SQS. Amazon distributes a library called the Amazon SQS Java Messaging Library, and it supports using SQS as the JMS messaging provider. However, it is only available if you are programming in Java, which leaves the C# and NodeJS developers out in the cold.

---

SNS (Simple Notification Service)

SNS is a way to send a message to a topic and then route the message to a number of notification mechanisms. The message can be routed simultaneously to one or more destinations. The destinations include:

• HTTP REST endpoint
• AWS Lambda Function
• SQS queue
• Email
• SMS

Unlike SQS, SNS does not have a dead-letter queue where it routes undeliverable messages. SNS is basically a fire-and-forget mechanism.

SNS messages are pushed to the destinations. The destination consumer does not have to worry about polling for messages.

In the SBS world, the RUN Event Dispatcher (RED) has some of the same functionality as SNS.

SNS-SQS Integration

You will notice that one of the delivery mechanisms that SNS supports is to push a message into an SQS queue. On the other side of the queue could be an application that can read the message and take some action. 

You can even push the single message to multiple SQS queues in order to execute some tasks in parallel.

Kinesis

IMPORTANT NOTE- Kinesis Streams is not available for the AWS Free Tier

Kinesis is the preferred hosted streaming platform for AWS. It differs from SQS and SNS in that Kinesis feels comfortable ingesting continuous streams of data, such as a stream of real-time stock quotes or a stream of signals from millions of IoT devices.

A Kinesis stream is subdivided into shards. Each shard can process a stream of data in isolation of other shards. This provides a degree of load-balancing. Each piece of data can contain a “partition key”, which directs that piece of data to be processed by a specific shard. 

Each Kinesis consumer has a shard iterator which is used to read data from the stream. In this sense, Kinesis is similar to Kafka. Since data is persisted in the stream, a consumer can retrieve data from the beginning of the stream. This supports the concept of “late joiners”, in which a new subscriber can retrieve all of the events that they might have missed. A side benefit of this is that it is easy to replay data for various testing scenarios.

Consumers run of EC2 instances. You can auto-scale consumers by hooking Kinesis up to Cloudwatch, and adding additional EC2 instances dynamically when needed.

Kinesis Analytics

AWS has a service that works with Kinesis that allows you to perform queries on the data in the stream as that data passes through the stream. This service is called Kinesis Analytics, and it gives Kinesis the kind of Complex Event Processing (CEP) capabilities that systems like Streambase and Esper have.

Kinesis Analytics uses a dialect of SQL to perform processing. You can use this capability to detect certain conditions and generate events, or use can use this capability to enrich or transform the data.

The streaming SQL code below detects a condition where the change in a stock price is over 1%. If this condition is detected, an event is generated and put into another stream. A consumer of the other side of this new event stream can send a message to a user or trigger some sort of algorithmically-based trade. 

CREATE OR REPLACE STREAM “INTERESTING_STOCK_EVENT_STREAM” 

    (ticker_symbol VARCHAR(4), sector VARCHAR(12), change DOUBLE, price DOUBLE);

CREATE OR REPLACE PUMP “STREAM_PUMP” AS 

   INSERT INTO “INTERESTING_STOCK_EVENT_STREAM”

      SELECT STREAM ticker_symbol, sector, change, price 

      FROM   “STOCK_QUOTE_STREAM”

      WHERE  (ABS(Change / (Price – Change)) * 100) > 1;

---

Apache Kafka

An alternative to the AWS-hosted messaging systems mentioned above is to provision your own EC2 servers and install and run Apache Kafka on those servers. 

I will not talk about the technology around Kafka here, as this has been discussed elsewhere. But I will talk about the differences between using Kafka on AWS and using one of the native AWS platforms.

Several articles point to Kafka being more performant that Kinesis for very high-throughput use cases. But if your application does not have the amount of streaming data that would compel you to use Kafka, then Kinesis is a simpler platform to use.

Some Pros for Kinesis

• Managed service
• Removes operational headaches and costs
  • Tuning Kafka can be a challenge, and Kafka engineers are difficult to find
• Costs can be lower than Kafka for a similar environment
  • With Kafka, you need hardware for the instances, for Zookeeper, for replication, and for data storage for retained messages
• Fits into the rest of the AWS stack seamlessly
• Consolidated monitoring via AWS CloudWatch
• Elasticity – we can bring up Kinesis when we need it.
• Scale-out transparently at times of heavy usage
• Kinesis Analytics add-on
• We avoid the fragile nature of the integration of Zookeeper and Kafka

Some Pros for Kafka

• No vendor lock-in
• Wide support for C# clients
  • Most Kinesis APIs are Java-based
• We do not have to pay for Kafka, but we would have to pay for the EC2 servers that host Kafka
• Kafka SQL gives Kafka some of the same capabilities as Kinesis Analytics
• Since Kafka is open-source and part of the Apache project, we have visibility into Kafka (bug fixes, roadmap)
• Supports wildcard subscriptions
• Integrated with other Apache projects like Spark, Storm, and Samza

As a happy medium between performance and a full-managed service, we can consider using a completely managed Kafka service that runs on AWS. This service is run by Confluent, who is a consultancy that specialized in Kafka and was founded by the original Kafka developers at LinkedIn.

 

Comparison

Feature	SQS	SNS	Kinesis	Kafka
Native AWS Service	Y	Y	Y	N
Chargeback model	Per-request plus data egress	Pushes and deliveries. Different pricing for different delivery methods.	Shards per hour	N/A
Push vs Pull	Pull	Push
Max message size	256K		1 MB	Configurable, but defaults to 1MB
Max message throughput	Unlimited for standard queue300 tps for FIFO		1000 PUT records per second per shard1 MB/sec input and 2 MB/sec output per shard
Message delivered to multiple consumers?	N	Y	Y
Message order preserved?	Only in FIFO queues	N	Y
Durable messages?	Messages are stored on multiple servers		Y
Replay of messages?	N	N	Y	Y
Data retention	60 seconds to 14 days	1-14 days (if not deleted)	1-7 days
Wildcard subscriptions?	N
Max queue depth	120K in-flight messages20K for FIFO queues		N/A	N/A
Dead-letter Queues?	Y		N
Encryption?	Y		Y
Scaling	Transparently auto-scale through Cloudwatch	Transparently auto-scale through Cloudwatch	You can increase the number of shards used but you need to pre-provision the shards
Monitoring through Cloudwatch?	Y	Y	Y
Language support for APIs	C#, C++, Java, NodeJS, Python, Ruby, PHP, Go	C#, C++, Java, NodeJS, Python, Ruby, PHP, Go	C#, C++, Java, NodeJS, Python, Ruby, PHP, Go	C/C++, Python, Go, Erlang, .NET, Clojure, Ruby, NodeJS, Perl, PHP, Rust, Java, Scala, Clojure, Swift

Feature	Kafka	Kinesis
Storage of Messages	As much as you want. On the cloud, you pay for storage.	24 hours by default. Up to 7 days with a config change
Ordering of Messages	Partition level	Shard level
Message Delivery Semantics	Kafka guarantees at-least-once delivery by default. Kafka supports exactly-once delivery in Kafka Streams	Kinesis Data Streams has at least once semantics
Replication	Use Confluence’s MirrorMaker to replicate a topic	All message automatically replicates to all 3 availability zones
Scaling	Add more partitions to a topic	API call to increase the number of shards
Partition/Shard Modification	Increase only and does not repartition existing data	Re-shard by merging or splitting shards
Partition/Shard Limitation	No limit. Optimal partitions depend on the use case	500 shards in US East (N. Virginia), US West (Oregon), and EU (Ireland) regions. 200 shards in all other regions.
Security	Either SSL or SASL and authentication of connections to Kafka Brokers from clients; authentication of connections from brokers to ZooKeeper; data encryption with SSL/TLS	Data can be secured at-rest by using server-side encryption and AWS KMS master keys on sensitive data within KDS. Access data privately via your Amazon Virtual Private Cloud (VPC)
Tools	Kafka Connect – gets data in and out of Kafka. Kafka Streams – stream processing of the data that flows through Kafka.	Video Streams Data Streams Data Firehose Data Analytics
Monitoring	Yammer Metrics for metrics reporting in the server	AWS CloudWatch and CloudTrail
Limitations	Default is 1MB per message, but can be changed	Message size is limited to 1MB. You can only get records 5 times per second and up to 2MB per shard. Need to add more shards (at an additional cost per shard) to scale.
Dependency	ZooKeeper	DynamoDB
Pricing	Per node No concept of elasticity	Per shard hour Per storage after 24 hours Per PUT request Shards are elastic, so you can decrease shards during slow periods