Refer to graphviz official documentation on how they should look. This is how they look when I try to change shape of one of the examples in racket-graphviz documentation:
(define dg (make-digraph
`(("Locked" #:shape "circle")
("Unlocked" #:shape "doublecircle")
(edge ("Locked:n" "Locked:n") #:label "Push")
(edge ("Locked" "Unlocked") #:label "Coin")
(edge ("Unlocked" "Locked") #:label "Push")
(edge ("Unlocked:n" "Unlocked:n") #:label "Coin")
(same-rank "Locked" "Unlocked"))))
(digraph->pict dg)

This is what I see with doubleoctagon instead of doublecircle

And tripleoctagon looks similar.
Any idea what might be happening?