In 2014, Google approached a startup called Docker proposing the two collaborate on software each was developing to help companies manage lots of complex applications, according to people with knowledge of the proposal. But Solomon Hykes, Docker’s founder and CTO, said no. He wanted to go it alone.

Three years later, the cost of Mr. Hykes’ previously unreported decision is becoming apparent. The software that Google was developing was Kubernetes, an open-source product that now dominates its segment of the cloud software market. Docker’s rival software, Swarm, is also open-source but isn’t anywhere near as popular, two former Docker employees say. Docker generates between $30 million and $50 million a year in revenue by selling Swarm packaged with security, networking and storage software, estimates Jay Lyman, an analyst at 451 Group.