"Brian Griffin was an important part of our viewing experience," a fan wrote on the Change.org petition