At least ZFS On Linux, which given the tags on the question is probably what you are referring to, does not support your proposed usage at present.
As I have discussed elsewhere, ZFS does not support removing constituent devices from a striped set.
The pool itself is a striped set of one or more vdevs, where the vdevs may be single device, mirror sets or raidzN redundant sets. (In the case you describe (JBOD), the vdevs would consist of single underlying storage devices.) For brevity, the vdev level is elided in the
zpool status and similar commands' output if there is only a single vdev in the pool, but it's still there and can be viewed for example using the ZFS data structures dumper/debugger
The effect of this is that once a vdev has been added to a pool, it cannot be removed except by recreating the pool from scratch and transferring the data from the old pool to the new pool.
The subject of ZFS "block pointer rewrite" (which in ZFS is needed for what you want to do) comes up every now and then, and I think it's slated to be implemented "at some point", but there are many issues that are higher priority and considerably easier to implement.
Remember that ZFS is designed and implemented as an enterprise file system. Because in enterprise settings doing what you propose would be seen as very nearly insane, I wouldn't think of it as likely that what you want to do will ever be officially supported; it might however at some point in the future become possible to do, much like using deleted sparse files as backing devices is a not-really-supported-but-still-often-suggested way of gradually increasing the effective redundancy level of raidzN vdevs.