The thing you want is "glue records" the upper level server would serve ns1.example.com (this is an approved domain for example use, better to use example.com than making your own example up) as the authoritative name server. Then provide the glue record which says "ns1.example.com is at IP address X".
It should ask for IP addresses as well as hostname. Otherwise they only assumed people will "host" their domain in another hosted, as opposed to self-hosting.
In that case (and in any other case) change your registrar to someone else who supports glue records.