One way to study early Christianity is to look at it through the eyes of its opponents. I've often noted that the early opponents of Christianity describe the religion in a manner that contradicts Roman Catholic claims about church history. It's instructive to observe what men like Trypho, Celsus, and Caecilius tell us about early Christianity and what arguments they bring against the religion. It's also important to notice how individuals like Justin Martyr, Origen, and Minucius Felix respond on behalf of Christianity. Those early interactions between Christians and their critics provide us with a lot of evidence against the claims of Catholicism (and Eastern Orthodoxy, etc.) on the veneration of images, prayer to the dead, and other issues.
One of the most substantial problems for Catholicism in the early responses to Christianity is the absence of any reference to a papacy. Remember, Catholics often tell us how important the papacy allegedly is. We're supposed to think that it's the foundation of the church, central to Christian unity, an infallible source of Christian teaching, and so on. We're told how valuable the papacy is in the abstract and how practical it is, how it's such a good idea on so many levels to have such an office. We're often pointed to John 17:21, as if Jesus was referring there to denominational unity under Roman Catholicism and the papacy. Well, if we need to submit to the papacy "so that the world may believe" (John 17:21), shouldn't we expect the world to know that the papacy exists and to refer to it when discussing Christianity? Isn't that especially true during the earliest centuries of church history, when, according to many Catholics, there was such widespread and consistent unity under the Pope? What if, instead of acknowledging such unity and responding to it along the lines of John 17:21 or focusing on the papacy as one of the most foundational issues to be addressed (as we'd expect if Christianity was Roman Catholic at the time), the early responses to Christianity show no knowledge that a papacy even exists and sometimes make comments suggesting its nonexistence?
Something worth considering in this context is what Celsus, a second-century pagan critic of Christianity, said about Christian disunity and how Origen, a Christian, responded. See here.
Neither Celsus and his Jewish sources nor the other earliest non-Christian sources who comment on the religion refer to a papacy. And we have much more than documents like Justin Martyr's Dialogue With Trypho and Origen's Against Celsus to go by. Larry Hurtado notes:
"Whatever their actual success, clearly some Christians made impressive efforts to disseminate their works, not only among fellow believers, but more widely as well. Here again, it is difficult to find an equivalent effort by other religious groups of the day….Philo of Alexandria's Embassy to Gaius might serve as a kind of precedent. But the sheer number of Christian apologia [apologetic] texts is unprecedented." (Destroyer Of The Gods [Waco, Texas: Baylor University Press, 2016], 132-3, n. 90 on 249)
Men like Aristides and Tertullian wrote apologies in which they responded to objections to Christianity and anticipated potential objections. They address the deity of Jesus, his virgin birth, his resurrection, the second coming, the inspiration of scripture, how to interpret various passages of scripture, Christian moral standards, the apostles, Christian teachers, the nature of the church, and many other topics. But they say nothing of a papacy, to explain it, defend it, anticipate objections to it, or anything else.