You could like make a simple accumulator machine out of logic gates and enter binary instructions expressed in hexadecimal into its register to program it, yeah, but it's not capable of all the operations of a computer. But yes the first programming was just op codes, switches flipped or punch cards, there was no assembly language. But assembly language is pretty much just mnemonics for operations and registers. Like I had to write a couple C programs in school and use GNU C compiler to disassemble them into x86 assembly and see what it was doing on that level, then we "wrote" some x86 assembly by copypasting a lot of instructions but its not that hard to make something that works in like x86 assembly or like Jasmin (Java virtual machine assembly language) if it's simple enough.