Help setting up an router as an access point

I dont even know if this is possible, but here is my situation, my main router is downstairs by the TV and the wifi doesnt work reliably in my bedroom upstairs, but it does work in my office upstairs. i have successfully in the past been able to share my wireless connection over ethernet from the back of my PC into another pc without wifi but never a router, my question is if i share my wireless connection over the wired connection on my pc can i connect that wired connection into an old router to turn it into another access point? hope this makes sense

if this isnt possible ill go get a powerline adapter and use that