In computer science, a stack is a last in, first out (LIFO)
This is how the STACK data structure is defined in computer science.
Say you have elements A, B, C... E and you are going to put them in to a an object with one side closed in order of A, B, C, D and finally E. So you can see A is on the bottom and E is on top. You can't take A out. If you need to get A out, first you need to take out E, then D, after that E and so on until you reach 'A'. In other words, you can only access the top most element (or last input). That's why we call the Stack a Last In, First out style (LIFO) data structure.

- 1.gif (1.96 KiB) Viewed 10343 times
that stack elements have a natural order
Explanation is also below that if you read carefully. Here, the access to last input is only possible in a stack. When we put an element we call that operation "PUSH". When we need to take one out we call "POP. Let's get an example from nature.
PUSH and POP are the only two operations permit to call on a Stack.
Say there are few heavy iron disks (you are only able to hold one at a time) and you store all disks on top of the other. Now, think whether you can take a middle disk? You can't because, you will have to lift few disks at ones. This is not possible as you can only lift one at a time. So the procedure is, you take the top one out, then the other one and do the operation until you reach the middle disk you want. Is that right. This is called the natural order in stack (a practical reality that you can find in the nature).
Elements are removed from the stack in the reverse order
Again consider my heavy iron disk example. You can't lift more than that, so you have to take each one from top until you reach the element you want.
Say the iron disks are numbered from A to E.
You call following operations.
PUSH ( A )
PUSH ( B )
PUSH ( C )
PUSH ( D )
PUSH ( E )
You get the elements as in the following picture.

- 1.gif (1.96 KiB) Viewed 10343 times
Now you need to take C out. You need to call as below.
POP (this will give you E)
POP (this will give you D)
POP (this will give you C.. Here comes what you wanted)
stack pointers
On the above image, see there is an arrow with label "Top" that is pointing to the top most element. In a stack, we always point at the top most element as all operations are restricted to that. This pointer is called the Stack pointer.
In that image, stack pointer is pointing to element 'E'. However if you call POP, then then E will be poped out and the stack pointer will be pointed to element 'D'.
Read more on "Stacks" which is taught as the first lesson under Data Structures & Algorithms.