The problem that occurs on your 3x3x3 Cube is called Void Parity. When you solve a Void Cube you can have everything solved, except for two swapped corners. This is due to the "centers" not being at the correct place.
So, why does this occur:
A 90° slice turn is a 4-cycle of edges and a 4-cycle of centres. A 4-cycle is an odd permutation (means an odd number of swaps is needed to reach the original state).
Your situation shows an odd parity of edges whereas the corners are solved (= even permutation; 0 swaps needed to reach the solve state).
Doing a 4-cycle (i.e. a 90° slice turn) will switch the parity of edges from odd to even. The parity of centres switches too (this is invisible on a Void Cube, but visible on your 3x3x3 Cube).
A sequence to solve the Void Parity on a Void Cube: R F' L2 F2 U' D R' F B' D' F2 D F'
If you do this algorithm on a solved Rubik's Cube, you'll see the single swap (i.e. an odd number of permutations) of edges AND a 4-cycle of centers (a second odd number of permutations of another piece group).
An odd parity of one piece group of a Rubik's Cube can only be seen, if another piece group shows an odd parity as well.
(Similarly a 90° face turn switches the parity of edges and corners at once.
Note that the term parity has to do with permutations of elements for twisty puzzles, and is unrelated to the similar mathematical term 'parity'. See the two posts below for what 'parity' means on twisty puzzles.)
In short: Any outer layer turns of the cube produce an even permutation - a 4-cycle of edges, and a 4-cycle of corners. On a regular 3x3, a slice move does a 4-cycle of edges and a 4-cycle of centers. However, on a void cube, it's just a 4-cycle of edges (an odd permutation). The way you ignore the centers while they are off by 90 degrees, causes the same issues as with a Void Cube.
If you want some more general information about twisty puzzle parities, I can refer to these two posts of the Cubers-reddit:
This doesn't answer the Void Cube parity in particular, but should give a more general understanding of what Twisty Puzzle Parity is, and why it occurs on some puzzles (like the 4x4x4 Cube).
Basically, the law of cubes state that:
Even permutation + Even permutation = Even permutation
, no problems here
Odd permutation + Odd permutation = Even permutation
, again no problems here. (This is the most common one, like the 4-cycle of corners AND 4-cycle of edges when you do a single 90 degree face-move.)
Even permutation + Odd permutation = Odd permutation
. Here you have an Odd Parity, or just parity for short. If you start with an even permutation (let's say a prankster twisted a single corner, or swapped two corners), it doesn't matter how many moves you make, all you can do is even permutations, and a single twisted corner or single swap is an odd (one twist / swap) permutation, so you'll never be able to solve it (unless you twist / swap them back manually).
This last one is what currently happens for you in your second picture. No matter the moves you make with the centers colors and face colors as you currently have, you won't be able to make a single swap of corners since it's an odd swap. (You can still solve the cube regularly, or in the Six Spots or Checkerboard pattern, but not in the configuration like that because of the odd swap it requires.)
PS: Your first picture is called the Six Spots pattern, which can be reached from a solved cube with the algorithm: U D' R L' F B' U D'
. :)