Wireless problems in large building; do I need Mesh Networking? Can I use existing Routers?

I've got a $100-150 Asus Router that was doing fine in a 3 bedroom townhouse a few years ago. I now live in a 5000 sq ft space across three floors, wherein the network is as follows:

1Gb/s Modem from the ISP -> Ubiquiti 4-port wired router -> Asus RT-AC56 wireless router

I now have too much space and too many devices for this configuration. Firstly, the 5Ghz band for the wireless router is operating on a different SSID, so LAN interaction with devices that don't support that frequency is an issue. Second, the connected device list has grown from:

1 Phone

1 Laptop

1 PC

to

4 Phones

2 Laptops

4 PCs

1 Multifunction Printer

2 Steam Links

1 Smart Television

2 Xbox 360s

I'm now in a bit of a loop. Firstly, I can only connect devices a certain way and through the Wireless SSID if I need them to interact. Secondly, and far more annoyingly, the more the router is in use, the more issues begin to arise. Granted, one could call that somewhat normal, but these aren't typical latency issues which should be occurring. All connected machines begin to receive more and more pixellated video, "stuttering" in online games, and the Steam Links start to have issues keeping their connection to the host PC (which is on a wired connection to the wireless router). I even get some errors from the links about trouble communicating with the controller, but I think that's passed on from the PC because of latency between it and the link.

Restarting the wireless router fixes it for about an hour before symptoms begin to recur.

Obviously, I'm making a terrier do the job of a sled husky, so I know an upgrade is in order. As I look into it, Mesh Networking seems like my solution, but I'm confused by it. How is this different from just getting a bunch of routers together and tossing them into repeater mode? If it is different, could I use an existing wireless router to connect to the mesh connectivity nodes? if I have a wired-only router that performs really well (the aforementioned ubiquiti) do I even need another router, or can I just start connecting up the access nodes to that? Finally, are there any security risks with mesh that aren't present in traditional AP networking?