​Building an in-memory real-time distributed data platform is a challenge and a passion that we have at Hazelcast. To create such a platform we used raw Java, our own RPC and concurrency stack, distributed primitives, replication and Raft-based consensus protocol. In this talk, on the example of our issues, we will discuss what you should care about while building a distributed system: what replication options you have, how messages reordering could resurrect the dead node, how harmful the retries can be, and finally, how one slow node can bring down the entire cluster even with the Raft consensus protocol.