I figured out it's a problem with family sharing. I have to log in as me, (the account that bought the game), then log in as him, and then it works.
To be clear, this IS a bug. But the details are complicated:
On Xbox One, anyone logged in on the device can play any game installed on it, even if the game's owner is not logged in. They do have to log in once to activate the installation, however.
This is independent of Microsoft family accounts that can be linked as parent and child. Though if you are linked, they can also play games you as a parent own on other Xboxes.