1. Books are expensice.
1-1. Books in general are expensive.
[Is this explanation correct?]
2. The books are expensive.
2-1. The specific books are expensive.
[Is this explanation correct?]
3. The books in this library are expensive.
3-1. All the books in this library are expensive.
[Is this explanation correct?]
4. Books in this library are expensive.
4-1. Some books in this library are expensive.
[Is this explanation correct?]
1 and 3 are correct.
Your explanation for 2 and 4 would be correct if you indicate which books are expensive.
The art books ...
The history books ...
etc.