Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have several files with names containing various Unicode characters. I'd like to rename them to only contain the "printable" ASCII characters (32-126).

E.g,

Läsmig.txt         //Before
L_smig.txt         //After
Mike’s Project.zip 
Mike_s Project.zip 

Or for bonus points, transcribe to the closest character

Läsmig.txt
Lasmig.txt
Mike’s Project.zip
Mike's Project.zip

Ideally looking for an answer that doesn't require 3rd party tools. (Edit: Scripts encouraged; I'm just trying to avoid niche shareware apps that need to be installed to work)


Power shell snippet that finds the files I'm interested in renaming:

gci -recurse | where {$_.Name -match "[^\u0020-\u007E]"}

Unanswered similar python question - http://stackoverflow.com/questions/17870055/how-to-rename-a-file-with-non-ascii-character-encoding-to-ascii

share|improve this question
1  
There’s no such thing as “extended ASCII”. –  kinokijuf Aug 24 '13 at 20:11
1  
@kinokijuf, tell that to everybody else. –  Synetech Nov 29 '13 at 20:04
1  
@kinokijuf, and of course, nothing existed before Windows NT. –  Synetech Nov 29 '13 at 22:50
1  
@kinokijuf, Synetech is correct. The extended ASCII code set had been in existence for over a decade when Windows NT shipped. Every DOS program known to man used the extended ASCII set. –  Roger Nov 30 '13 at 2:00
2  
@Kinokijuf, there are code pages now. That's not in dispute. DOS added code page support only in DOS 3.3. However, the extended ASCII character set was built into the ROM of the original IBM PC display adapters. See this site for more info. –  Roger Dec 19 '13 at 19:23

2 Answers 2

up vote 1 down vote accepted

I found a similar topic here on Stack Overflow.

With the following code most of the characters will be translated to their "closest character". Although i couldn't get the translated. (Maybe it does, i can't make a filename in the prompt with it ;) The ß also does not get translated.

function Remove-Diacritics {
param ([String]$src = [String]::Empty)
  $normalized = $src.Normalize( [Text.NormalizationForm]::FormD )
  $sb = new-object Text.StringBuilder
  $normalized.ToCharArray() | % {
    if( [Globalization.CharUnicodeInfo]::GetUnicodeCategory($_) -ne [Globalization.UnicodeCategory]::NonSpacingMark) {
      [void]$sb.Append($_)
    }
  }
  $sb.ToString()
}

$files = gci -recurse | where {$_.Name -match "[^\u0020-\u007F]"}
$files | ForEach-Object {
  $newname = Remove-Diacritics $_.Name
  if ($_.Name -ne $newname) {
    $num=1
    $nextname = $_.Fullname.replace($_.Name,$newname)
    while(Test-Path -Path $nextname)
    {
      $next = ([io.fileinfo]$newname).basename + " ($num)" + ([io.fileinfo]$newname).Extension
      $nextname = $_.Fullname.replace($_.Name,$next)
      $num+=1
    }
    echo $nextname
    ren $_.Fullname $nextname
  }
}

Edit:

I added some code to check if a filename already exists and add (1), (2) etc... if it does. (It's not smart enough to detect an already existing (1) in the filename to be renamed so in that case you would get (1) (1). But as always... everything is programmable ;)

Edit 2:

Here is the last one for tonight...

This one has a different function for replacing the characters. Also added a line to change unknown characters like ß and for example to _.

function Convert-ToLatinCharacters {
param([string]$inputString)
  [Text.Encoding]::ASCII.GetString([Text.Encoding]::GetEncoding("Cyrillic").GetBytes($inputString))
}

$files = gci -recurse | where {$_.Name -match "[^\u0020-\u007F]"}
$files | ForEach-Object {
  $newname = Convert-ToLatinCharacters $_.Name
  $newname = $newname.replace('?','_')
  if ($_.Name -ne $newname) {
    $num=1
    $nextname = $_.Fullname.replace($_.Name,$newname)
    while(Test-Path -Path $nextname)
    {
      $next = ([io.fileinfo]$newname).basename + " ($num)" + ([io.fileinfo]$newname).Extension
      $nextname = $_.Fullname.replace($_.Name,$next)
      $num+=1
    }
    echo $nextname
    ren $_.Fullname $nextname
  }
}
share|improve this answer
    
The ß also does not get translated. Probably because the eszett is supposed to get mapped to ss which is two characters. (Well either that or to B which would be dumb if you’re not trying to use 1337-speak.) There’s obviously no built-in mapping, so you would have to handle it separately. –  Synetech Nov 29 '13 at 22:29
    
Thank you for all the time you put into this. Works a treat. –  RJFalconer Jun 8 '14 at 18:12

I believe this will work...

$Files = gci | where {$_.Name -match "[^\u0020-\u007F]"}

$Files | ForEach-Object {
$OldName = $_.Name
$NewName = $OldName -replace "[^\u0020-\u007F]", "_"
ren $_ $NewName
}

I don't have that range of ASCII filenames to test against though.

share|improve this answer
    
You can easily create some test files with right-click → New Text Document then type some ASCII characters mixed with some extended ANSI/Unicode characters. –  Synetech Aug 24 '13 at 19:40
2  
I just ran a test with most permutations. Not surprisingly, it worked for the most part, but you might run into errors if the ASCII-only filenames conflict with existing filenames (which could also happen if other files are renamed, e.g., resumé1.doc, resumé2.doc, resumé.doc, etc.) –  Synetech Aug 24 '13 at 20:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.