Annuaire et Recherche Thématique
bugs.debian.org
-
- #380226 - NTFS (partition) not recreated correctly after resize:incorrect start sector - Debian Bug report logs :: resize partition
Apercu : Bug report received and forwarded. The two are completely similar, except that the first is successful and the second leads to corruption. The corruption only becomes clear _after_ the physical partition is resized too; resizing the partition back to its original size does not get the partition back. Versions of packages ntfsprogs depends on: ii libc6 2. Shared libraries ii libfuse2 2. Erspace library ii libntfs8 1. Extra info received and forwarded to list. Here is the correct output. Setting required flags on partition. Volume has unsupported flags set. File and it was not empty. Will not be able to remount read-write. Directory index record with vcn 0x0 is corrupt. Failed to find inode number for hiberfil. Will not be able to remount read-write. Directory index record with vcn 0x0 is corrupt. Extra info received and forwarded to list. The two are completely similar, except that the first is successful > > and the second leads to corruption. That is the same in both cases. The corruption only becomes clear _after_ the physical partition is > > resized too; resizing the partition back to its original size does > > not get the partition back. If the partition was mountable before changing the physical partition, > it is most likely that the bug is in the partitionning tool, and not > in ntfsresize. Vistas partitions, and it worked. He is in vocation at the moment and > can not reply. This one > is the most common cause for the problem at hand. Also note that it means that 2 major linux partitioning tools (fdisk and parted) would be wrong. It is always good to check with the latest version. Id appriciate if > you check with 1. Extra info received and forwarded to list. The two are completely similar, except that the first is successful > > > and the second leads to corruption. Windows no longer agreed to boot. Returning the partition table to what it was saved the day. This is because a cluster may contain several sectors, and the original starting sector may not be cluster-aligned. When the first sector moves, the tools and the kernel driver can no longer mount the partition. Thanks, that would be helpful. Extra info received and forwarded to list. Fledel wrote: > 20482843+ is different than 20481024. Windows no longer agreed to boot. Returning the > partition table to what it was saved the day. Windows certainly booted before resizing. So to me it seems irrelevant for this issue. Extra info received and forwarded to list. The two are completely similar, except that the first is successful and > the second leads to corruption. October, so there is a lot of room for possible improvements in 1. Extra info received and forwarded to list. Yuval was correct: using default cylinder based partitioning, the starting sector _is_ indeed changed from 2048 to 63. Thank you for helping find the cause of the issue. Extra info received and forwarded to list. Frans forgot to send the mail to control@b. Now the severity is correct. Extra info received and forwarded to list. As can be seen in the bug history, partman needs to create the new smaller/larger partition with the same start sector (not start cylinder) as the old partition. Extra info received and forwarded to list. As can be seen in the bug history, partman needs to create the new > smaller/larger partition with the same start sector (not start cylinder) > as the old partition. This clearly shows that resize_partition is called with correct parameters. This shows that ped_disk_set_partition_geom does _not_ create the partition at the requested sectors. The original starting sector 2048 has been changed to 63. Instead of the requested end sector 29298922, libparted sets it at 29302559. Conclusion: either resize_partition calls the wrong libparted function, or there is a bug in parteds ped_disk_set_partition_geom function. Extra info received and forwarded to list. Conclusion: either resize_partition calls the wrong libparted function, > or there is a bug in parteds ped_disk_set_partition_geom function. Extra info received and forwarded to list. Here is a recipe to reproduce the error. The procedure starts when the installer has been run up to hardware detection and the main menu is shown with partman selected. Next, run the attached test script. The script is just some snippets from partman put together, so it matches what actually happens in partman pretty well (in a somewhat simplified way of course). Extra info received and forwarded to list. The problem seems to be that parted explicitly aligns the partitions at cylinder boundaries. This call eventually calls msdos_partition_align() of libparted/labels/dos. A dos systems or something like that. A possible quick fix would to simply comment out the call to the bios alignment, like in the patch below. Im not sure if there are consequences for other (legacy) systems. This should probably be taken up with upstream; he might have a more satisfying solution to the problem. Fingerprint: c1f5 f24c d514 3fec 8bf6 | | bas@o2w. Extra info received and forwarded to list. Im unsure where else and how libparted is used though. Etch and it would be nice if someone else would try to either reproduce the bug or prove me wrong. Extra info received and forwarded to list. Im unsure where else and how libparted is used > though. Etch and it would be nice if someone > else would try to either reproduce the bug or prove me wrong. Im having a hard time distilling the information in these bug reports into a summary of what each bug is actually about or what the current status is. Having two bugs with the same bug title doesnt help matters any. Is this workaround implemented somewhere *other* than the linux-ntfs package? Ill try to work on providing one. Your rationale for ignoring 380226 also makes no sense to me; if this bug manifests in the installer, isnt that still a data loss bug that we need to fix before release? Can someone who understands whats going on with these bugs please fix up the bug titles so that theyre an accurate summary, to help the rest of us figure out which bits of information in the bug log are relevant to the current bugs? Extra info received and forwarded to list. Im unsure where else and how libparted is used > > though. Etch and it would be nice if someone > > else would try to either reproduce the bug or prove me wrong. Im having a hard time distilling the information in these bug reports into > a summary of what each bug is actually about or what the current status is. Bass suggested workaround, to see if it does cause problems on older systems? Extra info received and forwarded to list. Im having a hard time distilling the information in these bug reports > into a summary of what each bug is actually about or what the current > status is. Is this workaround implemented somewhere *other* than the linux-ntfs > package? Yes, it is implemented in partman, not ntfs-resize. It does not block resizing data partitions. The upstream maintainer of ntfsprogs only recently was convinced that the issue is real. Vista partitions, but other partitions seem to get created aligned on cylinder boundaries by default (or wed have seen a lot more bug reports). Your rationale for ignoring 380226 also makes no sense to me; if this > bug manifests in the installer, isnt that still a data loss bug that > we need to fix before release? Resizing a partition that is not on a cylinder boundary is scary anyway: fdisk will also do the wrong thing unless you remember to change the units it uses from cylinders to sectors (using its u command). If you feel the libparted bug should be fixed before the release, that is perfectly fine by me. However, it really is an upstream issue and there probably is a very good reason why that alignment check was originally added. Bas patch and hope for the best. Can someone who understands whats going on with these bugs please fix > up the bug titles so that theyre an accurate summary, to help the rest > of us figure out which bits of information in the bug log are relevant > to the current bugs? There is no actual bug in partman, but the two other bugs makes a workaround there necessary until the other two issues are fixed, hence the blocks. Vista partitions is small enough to ignore it (people should always backup their data before resizing anyway, right? Rs, so feel free to ask me for additional info if you need it. Extra info received and forwarded to list. Vista partitions, but other partitions seem to get > created aligned on cylinder boundaries by default (or wed have seen a > lot more bug reports). Your rationale for ignoring 380226 also makes no sense to me; if this > > bug manifests in the installer, isnt that still a data loss bug that > > we need to fix before release? Extra info received and forwarded to list. Extra info received and forwarded to list. Conclusion: either resize_partition calls the wrong libparted function, > or there is a bug in parteds ped_disk_set_partition_geom function. Looking at the definition of ped_disk_set_partition_geom() in libparted/disk. The constraint warning from ped_disk_add_partition() applies. The partitions geometry may be changed, subject to \p constraint. So, maybe this is not so much an issue of the ped_disk_set_partition_geom() function being actually wrong but rather calling it with the exact constraint? Extra info received and forwarded to list. Oswalds proding, i had a look at the history of this bug. A compatibility issues, which brang into memory the whole huge set of problems which where present 1-2 years ago with windowy filesystem resizing. Maybe otavio or xavier are the best candidates for that), to take contact with upstream, and not to try to solve the issue alone in debian, and possibly do something which may break the other compatibility issue affecting older hardware, which we may or not be able to detect before the etch release. Extra info received and forwarded to list. Im not sure what the right thing to do is at this time. This constraint is for partitions starting on the first cylinder. So it needs to start at the second track, and would mean in most cases be on sector 63 (63 sectors/track). However, vista seems to create the first one at sector 2048, which is in even in the middle of a track. Im currently not aware of. Extra info received and forwarded to list. This constraint is for partitions starting on the first cylinder. They | * must start on the 2nd head of the 1st cylinder. So it needs to start at the second track, and would mean in most cases | be on sector 63 (63 sectors/track). However, vista seems to create the first one at sector 2048, which is in | even in the middle of a track. Well, does it really matter _why_ they did it? All it should do is change the _ending_ sector. But if that is the case, then the partition table was broken already before parted had anything to do with it. Of course, this is a totally different situation than in the case that parted is creating a new partition from scratch. In such a case, of course, it _does_ make sense to write a sane parition table with partitions starting at cylinder boundaries only. In the case that a new partition is created, everything should stay as it is, and both beginning and ending sector should be aligned. Fingerprint: c1f5 f24c d514 3fec 8bf6 | | bas@debian. Extra info received and forwarded to list. Well, does it really matter _why_ they did it? All it should do is change the _ending_ sector. But if that is the case, then the > partition table was broken already before parted had anything to do with > it. Of course, this is a totally different situation than in the case that > parted is creating a new partition from scratch. In such a case, of > course, it _does_ make sense to write a sane parition table with > partitions starting at cylinder boundaries only. In the case that a new partition is created, everything > should stay as it is, and both beginning and ending sector should be > aligned. The problem here is how to decide if we should or not align it. Extra info received and forwarded to list. In the case that a new partition is created, everything > > should stay as it is, and both beginning and ending sector should be > > aligned. The problem here is how to decide if we should or not align it. Thats > the most difficult question. Fingerprint: c1f5 f24c d514 3fec 8bf6 | | bas@debian. Extra info received and forwarded to list. The problem here is how to decide if we should or not align it. Thats >> the most difficult question. Ill check what we can do. Extra info received and forwarded to list. The problem here is how to decide if we should or not align it. Thats >> the most difficult question. Microsoft is using 2048 sectors/track instead of 63. Extra info received and forwarded to list. The problem here is how to decide if we should or not align it. Thats > >> the most difficult question. Microsoft is using 2048 sectors/track > instead of 63. Extra info received and forwarded to list. This seems like the wrong thing to do. Extra info received and forwarded to list. This seems like the wrong thing to do. We are speaking about th end sector here though. Extra info received and forwarded to list. Microsoft is using 2048 sectors/track > > instead of 63. Afaik, the version is still 3. Frans probably knows more about this. But that would still leave the question on how to not change the start sector without breaking something else. Extra info received and forwarded to list. Frans probably knows more about this. Extra info received and forwarded to list. Frans probably knows more about this. Ah, didnt remember that part. Extra info received and forwarded to list. Microsoft is using 2048 sectors/track > > > instead of 63. Afaik, the version is still 3. Frans probably knows more about this. But that would still leave the question on how to not change the start > sector without breaking something else. You can set constraints on both the start and the end, if i remember well. Extra info received and forwarded to list. Vista has any problem with non-2048 sector boundaries havnet tried it though). The problem here seems to be that the partition boundary is changed without changing/moving the fs to reflect the new partition size. Fingerprint: c1f5 f24c d514 3fec 8bf6 | | bas@debian. Extra info received and forwarded to list. Bug#380226: fixed in parted 1. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to 380226@bugs. This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing ftpmaster@debian. Extra info received and forwarded to list. Bug reopened, originator not changed. Bug marked as found in version 1. Extra info received and forwarded to list. Im not sure what you are using libparted with. What software are you using that invokes libparted? Extra info received and forwarded to list. Frans is a starting sector relocation and it does indeed looks wrong to me. Extra info received and forwarded to list. First of all, thank you for your interest in helping solve this. Im not sure what you are using libparted with. What > software are you using that invokes libparted? We use ntfsresize to resize the file system. Windows for some reason no longer creates the partition starting on the second sector of the disk and that libparted, because of the current constraints, changes the starting sector of the physical partition which, of course, makes the file system inaccessible. The software we use for this in the installer is a fairly complex beast called partman, which consists of various components. Most of it is scripted, but the interaction with libparted is done through a program called parted_server. Extra info received and forwarded to list. First of all, thank you for your interest in helping solve this. Apologies for it taking so long. Im not sure what you are using libparted with. What > > software are you using that invokes libparted? We use ntfsresize to resize the file system. Windows for some > reason no longer creates the partition starting on the second sector of > the disk and that libparted, because of the current constraints, changes > the starting sector of the physical partition which, of course, makes the > file system inaccessible. Vista has changed that to accomodate large disks. Vista now aligns to sector 0x800 by default (2048), but its actually configurable by the administrator. Unfortunately, it will only present more problems for people like you and me down the road. The best we can do is accomodate other operating systems as much as possible. The software we use for this in the installer is a fairly complex beast > called partman, which consists of various components. Most of it is > scripted, but the interaction with libparted is done through a program > called parted_server. But you cannot do a partition resize since resize via parted(8) is intended to resize the filesystem and partition in one step. But you can remove the partition and recreate it with a higher end boundary, which effectively resizes the physical partition for you and then you can run ntfsresize on that. Does any of that help? Node in #parted and various other places. Extra info received and forwarded to list. Apologies for it taking so long. Debian > for this to work. It no longer changes the starting sector to 63, but also does not fix the problem (i. Otavio to revert that patch. From my reading of the relevant code (libparted/labels/dos. Extra info received and forwarded to list. Ive seen no response to that yet. Davids patch applied is correct. Bastian seems to agree with that. Im not at all impressed by the way this issue has been handled on the parted side over the past months. Ive have spent a couple of _weeks_ in total on both the ntfsresize and the parted issue up till now, providing solid information and testing since the very start. Debian and upstream) of parted would take bugs that screw up their users systems a bit more seriously. Extra info received and forwarded to list. Friday and discussed it in person. Then it combines them using ped_constraint_intersect() and passes the combined constraint into ped_disk_set_partition_geom(). However, this will disable resizing of any partition if the filesystem resizing is not supported. So the second step would be to change ped_file_system_get_resize_constraint() to return the correct constraint where the filesystem does not support resizing, as stated in the problem summary. Is there a way to make the kernel look for partitions on loopback devices? Windows, people just stare . Extra info received and forwarded to list. Then it combines them using ped_constraint_intersect() and passes the > combined constraint into ped_disk_set_partition_geom(). Absolute power is kind of neat. Extra info received and forwarded to list. This is not a bug in libparted; parted_server is simply not specifying the correct constraint for the resize operation. We cannot resize or move the fs but we can move the + * end of the partition so long as it contains the + * whole fs. This should be the case if its not a virtual partition (what does that mean? Absolute power is kind of neat. Extra info received and forwarded to list. Thanks for working on this issue. This is not a bug in libparted; parted_server is simply not specifying > the correct constraint for the resize operation. English how you arrived at that conclusion and what the patch you propose is supposed to do? It still needs to be run inside the installer though. Ill give your patch a try myself too. Extra info received and forwarded to list. Vista partition, and the patch works well. Vista boots correctly after the resize operations. It turns out that a minor additional patch in partman-partitioning is necessary (a short sleep to allow the system to settle before running the final ntfsresize -f command). This will help to decide how much additional testing is needed. Extra info received and forwarded to list. Vista partition, and the patch works well. Vista boots correctly > after the resize operations. It turns out that a minor additional patch in partman-partitioning is > necessary (a short sleep to allow the system to settle before running the > final ntfsresize -f command). This will help to decide how much additional testing is > needed. We cannot resize or move the fs but we can move the 37:+ * end of the partition so long as it contains the 38:+ * whole fs. If we care about the filesystem on the partition, we do not proceed unless we know what constraint it imposes on resizing. This can fail due to: 1. Failure to get the constraint from an open filesystem (line 19). This is a new possibility for failure; currently we would continue at the risk of data loss. Failure to get the filesystem type (line 27). This already results in failure (line 8). Failure to get the filesystem geometry (line 30). This is a new possibility for failure; currently we would continue at the risk of data loss. Out of memory (lines 27, 30, 44). There is a (very) marginally higher memory requirement. Quantity is no substitute for quality, but its the only one weve got. Extra info received and forwarded to list. It tries to probe even if open_filesystem is false (already identified). It can return after maximize_extended_partition() without rolling that change back. It doesnt check whether the existing invalid filesystem geometry returned by the probe is valid. Filesystem is already outside the partition + * so assume it is broken and do not constrain + * resizing. We cannot resize or move the fs but we can + * move the end of the partition so long as it + * contains the whole fs. Quantity is no substitute for quality, but its the only one weve got. Extra info received and forwarded to list. It tries to probe even if open_filesystem is false (already > identified). It can return after maximize_extended_partition() without rolling > that change back. It doesnt check whether the existing invalid filesystem geometry > returned by the probe is valid. This patch again results in the start of the ntfs partition being moved to sector 63. Extra info received and forwarded to list. It tries to probe even if open_filesystem is false (already > > identified). It can return after maximize_extended_partition() without rolling > > that change back. It doesnt check whether the existing invalid filesystem geometry > > returned by the probe is valid. This patch again results in the start of the ntfs partition being moved > to sector 63. In the resize_partition() function these are distinguished by the open_filesystem flag which implied to me that in the latter case we wouldnt expect to find a filesystem in the partition at all. Clearly thats not the case here. This patch reduces the difference that the open_filesystem flag makes. We cannot resize or move the fs but we can + * move the end of the partition so long as it + * contains the whole fs. It should be clear that all the new code (aside from the error check) only runs in the currently broken case, so this does not affect resizing ext2 etc. And none of it is running below maximize_extended_partition(). It should be possible for the user to work around that by deleting and recreating the partition rather than resizing it. However theres no obvious error message when this happens. Extra info received and forwarded to list. In the resize_partition() function these > are distinguished by the open_filesystem flag which implied to me that > in the latter case we wouldnt expect to find a filesystem in the > partition at all. Clearly thats not the case here. Could it be that a virtual partition is one that has been created or modified, but has not yet been committed to disk? This happens quite a lot in partman. It should be clear that all the new code (aside from the error check) > only runs in the currently broken case, so this does not affect > resizing ext2 etc. And none of it is running below > maximize_extended_partition(). This new patch works again. Watson if he can review your patch. Extra info received and forwarded to list. This new patch works again. Watson if he can review > your patch. During the tests only one thing showed up: the end of the partition is (not always? Partition 1 does not end on cylinder boundary. Ben has suggested a way to debug that. Ill try to do so. Extra info received and forwarded to list. In the resize_partition() function these > > are distinguished by the open_filesystem flag which implied to me that > > in the latter case we wouldnt expect to find a filesystem in the > > partition at all. Clearly thats not the case here. Could it be that a virtual partition is one that has been created or > modified, but has not yet been committed to disk? This happens quite a > lot in partman. Specifically, a virtual partition is one whose geometry does not match any of the partition geometries existing on disk. You can modify things like a partitions mount point without rendering it virtual. Resizing a partition is (at present) committed immediately and so doesnt produce a virtual partition, at least not at any time when the rest of partman is in a position to notice. However, if youve just created a partition in partman and then resize it, there is clearly no filesystem there and it can be resized simply by changing the geometry of the partition in parted_server (and the associated files in /var/lib/partman/devices) without having to worry about any filesystem resizing. Im currently contemplating handling ext2 and ext3 the same way since parted cant handle the resize_inode feature on those filesystems, but thats another story). Thus, open_filesystem is to be interpreted as a verb in the imperative voice; if its false, resize_partition is being instructed not to attempt to ped_file_system_open the filesystem (a prerequisite for ped_file_system_resize). The answer is that theres some unbelievably nasty code in partman-partitioning/resize. Some of this should probably end up as comments in the code. It should be clear that all the new code (aside from the error check) > > only runs in the currently broken case, so this does not affect > > resizing ext2 etc. And none of it is running below > > maximize_extended_partition(). This new patch works again. Watson if he can review your > patch. Watson [cjwatson@debian. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to 380226@bugs. This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing ftpmaster@debian. Correct mode in mkfifo call. Extra info received and forwarded to list. This is a test mail.
Voir #380226 - NTFS (partition) not recreated correctly after resize:incorrect start sector - Debian Bug report logs
