Austin Kelley’s new novel is a philosophical detective story in which the objective, verifiable world is a hypothesis, not a given.