Each user connects to a single server, this is their homeserver. Users are able to participate in rooms that were created on any Matrix server since each server federates with other Matrix servers. This means you can talk to anyone on any server. It also means you can host your own server, giving you control over all of your data. Self hosting also gives you the ability to customize your server to fit your needs including giving you the ability to bridge to other chat networks (such as IRC, XMPP, Discord, Telegram, etc) or to host bots.
Each message that is sent in a room is synchronized to all of the other servers that participate in that room. If one server goes offline, everyone else in the room can continue talking. Once that server comes back online it will be sent all of the messages that it missed while it was down.
Did we mention it is secure? Your private conversations can be secured by end to end encryption so the server has no idea what you are talking about.