We introduce the elliptic quantum toroidal algebra $U_{q,t,p}(\mathfrak{gl}_{1,tor})$. After giving some representations including a representation realized by using the elliptic Ruijsenaars difference operator, we construct intertwining operators of the $U_{q,t,p}(\mathfrak{gl}_{1,tor})$-modules w.r.t. the Drinfeld comultiplication. We then show that $U_{q,t,p}(\mathfrak{gl}_{1,tor})$ gives a realization of the affine quiver $W$-algebra $W_{q,t}(\Gamma(\widehat{A}_0))$ proposed by Kimura-Pestun. This realization turns out to be useful to derive the Nekrasov instanton partition functions, i.e. generating functions of the $\chi_y$- and the elliptic genus of the instanton moduli space, of the 5d and 6d lifts of the 4d ${\cal N}=2^*$ gauge theories and provide a new Alday-Gaiotto-Tachikawa correspondence.