No such space exists. We actually get the stronger statement that every isomorphism $\operatorname{End}(X) \stackrel\sim\to \operatorname{End}(\mathbf R)$ is induced by an isomorphism $X \stackrel\sim\to \mathbf R$ (unique by Observation 1 below). In contrast, in Emil Jeřábek's beautiful construction in this parallel post there is an 'outer automorphism' $\operatorname{End}(X) \stackrel\sim\to \operatorname{End}(X)$ that does not come from an automorphism $X \stackrel\sim\to X$ of topological spaces (it comes from an anti-automorphism of ordered sets).

I will use the substantial progress by YCor and Johannes Hahn, summarised as follows:

**Observation 1** (YCor). For every topological space $X$, the map $X \to \operatorname{End}(X)$ taking $x$ to the constant function $f_x$ with value $x$ identifies $X$ with the set of left absorbing¹ elements of $\operatorname{End}(X)$.

In particular, an isomorphism of monoids $\operatorname{End}(X) \stackrel\sim\to \operatorname{End}(Y)$ induces a bijection $U(X) \stackrel\sim\to U(Y)$ on the underlying sets.

**Observation 2** (Johannes Hahn). If $\operatorname{End}(X) \cong \operatorname{End}(\mathbf R)$, then $X$ is $T_1$. Since the closed subsets of $\mathbf R$ are exactly the sets of the form $f^{-1}(x)$ for $x \in \mathbf R$, we conclude that these are closed in $X$ as well, so the bijection $X \to \mathbf R$ of Observation 1 is continuous.

(The asymmetry is because we used specific knowledge about $\mathbf R$ that we do not have about $X$.)

To conclude, we prove the following lemma.

**Lemma.** *Let $\mathcal T$ be the standard topology on $\mathbf R$, and let $\mathcal T' \supsetneq \mathcal T$ be a strictly finer topology. If all continuous maps $f \colon \mathbf R \to \mathbf R$ for $\mathcal T$ are continuous for $\mathcal T'$, then $\mathcal T'$ is the discrete topology.*

Note that Observation 2 and the assumption $\operatorname{End}(X) \cong \operatorname{End}(\mathbf R)$ imply the hypotheses of the lemma, so we conclude that either $X = \mathbf R$ or $X = \mathbf R^{\operatorname{disc}}$. The latter is clearly impossible as it has many more continuous self-maps.

*Proof of Lemma.* Let $U \subseteq \mathbf R$ be an open set for $\mathcal T'$ which is not open for $\mathcal T$. Then there exists a point $x \in U$ such that for all $n \in \mathbf N$ there exists $x_n \in \mathbf R$ with $|x - x_n| \leq 2^{-n}$ and $x_n \not\in U$. Without loss of generality, infinitely many $x_n$ are greater than $x$, and we can throw out the ones that aren't (shifting all the labels, so that $x_0 > x_1 > \ldots > x$). Up to an automorphism of $\mathbf R$, we can assume $x = 0$ and $x_n = 2^{-n}$ for all $n \in \mathbf N$. Taking the union of $U$ with the usual opens $(-\infty,0)$, $(1,\infty)$, and $(2^{-n},2^{-n+1})$ for all $n \in \mathbf N$ shows that
$$Z = \big\{1,\tfrac{1}{2},\tfrac{1}{4},\ldots\big\}$$
is closed for $\mathcal T'$. Consider the continuous function
\begin{align*}
f \colon \mathbf R &\to \mathbf R\\
x &\mapsto \begin{cases}0, & x \leq 0,\\ x, & x \geq 1, \\ 2^nx, & x \in \big(2^{-2n},2^{-2n+1}\big], \\ 2^{-n}, & x \in \big(2^{-2n-1},2^{-2n}\big].\end{cases}
\end{align*}
Then $f^{-1}(Z)$ is the countable union of closed intervals
$$Z' = \bigcup_{n \in \mathbf N} \big[2^{-2n-1},2^{-2n}\big] = \big[\tfrac{1}{2},1\big] \cup \big[\tfrac{1}{8},\tfrac{1}{4}\big] \cup \ldots.$$
By the assumption of the lemma, both $Z'$ and $2Z'$ are closed in $\mathcal T'$, hence so is the union
$$Z'' = Z' \cup 2Z' \cup [2,\infty) = (0,\infty),$$
and finally so is $Z'' \cup (-Z'') = \mathbf R\setminus 0$. Thus $0$ is open in $\mathcal T'$, hence so is every point, so $\mathcal T'$ is the discrete topology. $\square$

¹Elements $f$ such that $fg = f$ for all $g$. (I would probably have called this right absorbing!)