A shell like the bash or command.com (up to Windows ME) or CMD.EXE (in later versions) provides an interface that (among other things) accepts commands from the user. What does an operating system look like before a shell is run? How were systems used before the first shell was developed (e.g. UNIX in the early 1970s)? If a computer cannot even accept commands (there is no command line), how can a user interact with it? What is this most basic interface? Can I run this interface in a terminal emulator or is there no way going behind a shell?
What does an operating system look like before a shell is run?
Depends on the OS and how you configure it. Linux can be configured to write boot text to a console device, whether a text mode console, framebuffer console or a serial port. It can also be configured to be perfectly silent. Some OS's/systems may write diagnostic information to a non-volatile memory that can be accessed by putting the system in developer, debug, or diagnostic mode. Many operating systems support outputting boot and diagnostic information to some form of UART, which may somehow be available on the unit even if hidden from the user (google "Add serial port to DD-WRT" for examples of where manufacturers hide serial ports and how you can get to them).
An OS doesn't have to have an external display at all - it's just another device to the OS.
How were systems used before the first shell was developed (e.g. UNIX in the early 1970s)?
Essentially (and leaving out a lot but this should get you the idea) - You loaded your program, either by flipping switches on a panel or using a paper-tape reader (these devices would write to memory directly without CPU intervention) and then start the CPU with another switch. The CPU would run this program, generate its output, and stop. This is batch processing as opposed to interactive processing. If you wanted to run a different program you had to do this over.
If a computer cannot even accept commands (there is no command line), how can a user interact with it?
I am no expert in this area but old, old computers like the Altair, IMSAI, and PDP-8 and such had front panel switches that directly controlled the CPU and could directly read and write memory without CPU intervention.
What is this most basic interface?
I believe most if not all modern CPUs have a "JTAG port" which allows the same type of direct operations. Keep in mind that for a long time most computers have been expected to have ROM or firmware that takes control of the system when it is turned on before it hands it off to an OS. Here, pre-boot utilities can exist, or a minimal mechanism for loading such utilities exists. Some bootloaders such as U-Boot can be accessed via the serial port. Bootloaders don't run "behind" the OS, they load the OS, hand control to it, and then they are no longer running.
Can I run this interface in a terminal emulator or is there no way going behind a shell?
No, you need a JTAG interface. That's diving into the realm of electronics and I admit I don't know very much about it, except that my GuruPlug comes with one and I can directly program the flash chip on the GuruPlug's board with it - meaning if something kills the bootloader on the GuruPlug, I have a "CPU independent" way of flashing it back.
An operating system doesn't have to provide a shell as the term is commonly used today (meaning an application which will accept commands interactively from the user), but such an operating system won't really "look like" anything at all to the user. As Oliver Salzburg mentions, it would probably just show a blank screen (if it has display output support at all), though there's no reason it'd have to. Take for example the Linux kernel's diagnostic output during the boot and kernel initialization process.
The shell, whether a graphical shell, command line interpreter or something else, uses the facilities provided by the operating system to do things like read commands, launch processes, do I/O redirection and so on.
However, there's no reason why the application that uses those facilities has to be a shell.
In the old days, operating systems were simply a collection of those "useful routines" that each application otherwise would have to rewrite from scratch, and computers were essentially batch processing devices. Things like file, disk and printer I/O were probably among the first to be collected into what is now known as an operating system, followed by process scheduling (it's worth noting that the early-1960s Apollo Guidance Computer was a multitasking computer). Applications could then make calls to the OS instead of using their own routines to do such things, which reduced programming complexity and probably helped reduce code size or execution time (because those system facilities could be heavily optimized and debugged once after which everyone would benefit). As computers became more and more common, operating systems added features which were largely user-centered, such as a way for the user to interact with the computer and give commands in an interactive fashion; graphical shells are simply an extension of that line of reasoning.
Also, not that long ago (think up to the late 1980s), it was still fairly common to write applications to run on the bare personal computer hardware, without the assistance of any ordinary operating system. This was particularly useful for games, as it avoided the memory and processing overhead of the operating system proper, but I'm sure there were other examples as well. In those cases, to some extent, the application was its own operating system and by consequence, the user interface provided by that application was the shell.
Early computers didn't have an OS in the sense we use the word today. They exposed functions implemented in hardware directly to any program running on it. There was only ever one program running on them at the same time. The program itself had to control all tasks, nothing was done 'in the background' by an OS.
But still there was an entry point for the user to start a program. If you stretch the word, you could call this a "shell". But basically, it was just the hardware waiting for the user to input the first bit of the program to be run. Be it in the form of pressed buttons, flicked switches, connected wires on a switchboard, punchcards, punched film or magnetic tape. Maybe they could even choose from several program options loaded earlier. But even selecting from a list displayed as glowing lights by pressing a button next to it can already be considered a 'shell'.
So if your definition of 'shell' is 'an interface that accepts commands from a user', then there was no time before it, at least for devices that you would even vaguely call a computer.
You might want to check out the pretty good wikipedia page about the history of computing, although it doesn't focus much on the input/output perspective.
It'll probably be a blank screen.
The shell is probably named the shell because it is a shell around the kernel (the kernel being the operating system). So, in a sense, it is the user interface for the operating system, whatever interface that is.
If an operating system only had one single function, which would be to shut down the computer, and you would build a program that accepts any keyboard input and then invokes that function, then that is the shell. There is no need to have a complex command line interface to build something that interfaces with the user.
From a historical view, I would guess that there were Punched cards ( https://en.wikipedia.org/wiki/Punched_card). To interact with a computer system. But I guess this is to far back for your question.
The first OS I used (1981) after college was PRIMOS on a Prime minicomputer. This was a time-sharing operating system and it supported a number of users, each using a terminal connected to the computer through an RS232 cable. You had to log on to the terminal giving a username and password. Your terminal session was a shell of sorts. You could edit files, compile, run all those things we do nowadays. All the terminals had access to the same filing system. Largely these terminals were no more than typewriters - no WYSISWYG editors, even emacs was a dream.
The operating system was structured much as they are now and could be imagined as a layer of onion skins. The innermost layer had very low level functionality - like hardware control, memory access. Going outwards, the file system would be added and then the user layers. In your program you would be able to interact with some layers, but not with the innermost ones (so you wouldn't be able to mess with the time-sharing or the actual hardware - lights and so on). A computer without a shell would be like one of these inner layers, it might be able to access a hard disk and a tape reader (really!), but it wouldn't know about files or users.
To boot an early computer you needed to load a basic set of instructions (this might involve toggling physical switches in a set sequence on the computer). This sequence would initiate a second small program, which might enable the tape reader to work. Then you'd load a more complex system through the tape reader, which might bring the disk drive on line. You could imagine a shell-less OS as being like one of these initial loaders.
Nowadays computers have similar sequences, so when you start the machine, it loads its BIOS, which starts the disks, network cards, video card and so on, then the BIOS looks for a specific program on the hard disk and runs that (on Windows at least). Unix does something similar it progressively sets up the kernel starting with very basic modules and building on them until it gets to the login prompt
Operating system as a definition is rather ambiguous. Is it a kernel itself? Is it the kernel and also accompanying tools? Linux is the kernel but GNU/Linux is the operating system based on Linux kernel and GNU project tools. Shell is an integral part of such operating system.
However, once Linux is up and finished with "booting" you need to tell it what program to run. From then on everything is in hands of that particular program. By default it is
init which knows what to do next and might eventually bring you to nice GUI login screen. But it doesn't have to be that way. You may boot like this
kernel /boot/vmlinuz-2.6.30 root=/dev/sda1 ro init=/bin/fancy
/bin/fancy will print "Hello world!". We don't need shell for that. If you would like to start some other program just spawn a new process by
man 2 fork and
man 2 execve, or read from terminal device and accept user's input. Still no need for shell.
In my view operating system shell is a program able to read user's input and then start another programs. If you think of it, it's fairly obvious why one would like to write such a program.
Even if you don't need to read user's input interactively, it is much more convenient to write simple shell script for your task. In that case it's just interpreter of stored shell commands. You may as well write your program in interpreter of some other language.
Operating systems without command line shells or graphical interfaces have many other kinds of "faces".
Embedded systems just do their job without any user interface, like the engine management system in your car (which you can get at in some ways via the OBD2 interface and a suitable terminal). Or may have digital keypads, knobs or whatever (think: microwave oven, elevator, or the faceplate of a modern sound system). Whether you regard these as a form of shell is subjective, of course.
Here is a high level recipe of how, as a hobbyist, you can make a useful computer without a shell on it:
- Build a circuit board for a microcontroller, or obtain a generic board.
- Hook it up to drive some useful devices, like, say, a moisture sensor, and a water valve. The microcontroller has peripheral pins for this purpose: UARTs, GPIO's, and so forth.
- Write the firmware to monitor the sensor, and water the soil.
- Upload the firmware through the development tools (So, no shell is required on the host computer to load and run anything, and the firmware is stored in flash memory on the chip). Programming may involve the microcontroller being plugged into a special programming board, or there may be a way of doing it while it is sitting in your actual board. The board interfaces to your PC (nowadays via USB, for instance) and you use some tools: like an IDE, or command line tools on the host.
- Deploy the thing: it's now essentially an electronic black box that somehow monitors soil moisture and turns on water, with no other inputs or outputs.
Very early general-purpose computers without shells had some means of input, like reading punched cards. But of course, punched cards had to be distinguished: does a punched card give the computer some special command, or is it a piece of a Fortran program? So "job control languages" had to be developed, which are de-facto command lines.
Before punched cards and job control languages, programmers had to toggle switches in order to feed binary code into the machine. You may have heard stories from old-timers who had to "toggle in" the instruction sequence to initiate a bootstrap. This is still done today, in a way: devices still exist which have jumpers or DIP switches, and there are some advantages to these compared to settings in firmware.
The first computer I was stuck on was a Siemens 305 in 1977, I was learning FORTRAN IV. It had a typewriter to run commands and print diagnostic message on paper, a punchcard reader, punchtape reader/writer and a printer. Not to forget the 40 MB removeable harddisk, 16 inch or so. So it already had a shell, and the interface was the typewriter.
I remember running a debugger and text editor (DDT and emacs) as the equivalent of a command shell back in the 1970's. So if you run emacs in console mode with any other program executed via eshell to a debugger that executes the program, you'll have a close experience.
Note that emacs has a complete lisp environment, so in addition to good command history editing and multiple virtual terminals, you have a very capable macro and scripting facility built in.