NSLayoutManager hides newlines no matter what I do

As far as I know, the default implementation of setNotShownAttribute: of the NSTypesetter of this class does not change the already generated glyphs in its glyph store. So calling super will have no effect. I just had to manually replace the glyphs before calling super.

So the most efficient implementation to display invisible characters (you will see the difference when you zoom in on the view) is:

Limitations of this approach: If your app has to have multiple fonts in the text view, this approach might not be a good idea, since those displayed invisible characters will also have different fonts. That's not what you might want to achieve.

1. Subclass NSLayoutManager and override setGlyphs to display space characters:

   public override func setGlyphs(_ glyphs: UnsafePointer<CGGlyph>, properties props: UnsafePointer<NSGlyphProperty>, characterIndexes charIndexes: UnsafePointer<Int>, font aFont: Font, forGlyphRange glyphRange: NSRange) {
       var substring = (self.currentTextStorage.string as NSString).substring(with: glyphRange)
   
       // replace invisible characters with visible
       if PreferencesManager.shared.shouldShowInvisibles == true {
           substring = substring.replacingOccurrences(of: " ", with: "\u{00B7}")
       }
   
       // create a CFString
       let stringRef = substring as CFString
       let count = CFStringGetLength(stringRef)
   
       // convert processed string to the C-pointer
       let cfRange = CFRangeMake(0, count)
       let fontRef = CTFontCreateWithName(aFont.fontName as CFString?, aFont.pointSize, nil)
       let characters = UnsafeMutablePointer<UniChar>.allocate(capacity: MemoryLayout<UniChar>.size * count)
       CFStringGetCharacters(stringRef, cfRange, characters)
   
       // get glyphs for the pointer of characters
       let glyphsRef = UnsafeMutablePointer<CGGlyph>.allocate(capacity: MemoryLayout<CGGlyph>.size * count)
       CTFontGetGlyphsForCharacters(fontRef, characters, glyphsRef, count)
   
       // set those glyphs
       super.setGlyphs(glyphsRef, properties:props, characterIndexes: charIndexes, font: aFont, forGlyphRange: glyphRange)
   }
   

2. Subclass NSATSTypesetter and assign it to your NSLayoutManager subclass. Subclasses will display newlines and ensure that each invisible character will be drawn in a different color:

   class CustomTypesetter: NSATSTypesetter {
   
       override func setNotShownAttribute(_ flag: Bool, forGlyphRange glyphRange: NSRange) {
           var theFlag = flag
   
           if PreferencesManager.shared.shouldShowInvisibles == true   {
               theFlag = false
   
               // add new line glyphs into the glyph storage
               var newLineGlyph = yourFont.glyph(withName: "paragraph")
               self.substituteGlyphs(in: glyphRange, withGlyphs: &newLineGlyph)
   
               // draw new line char with different color
               self.layoutManager?.addTemporaryAttribute(NSForegroundColorAttributeName, value: NSColor.invisibleTextColor, forCharacterRange: glyphRange)
           }
   
           super.setNotShownAttribute(theFlag, forGlyphRange: glyphRange)
       }
   
       /// Currently hadn't found any faster way to draw space glyphs with different color
       override func setParagraphGlyphRange(_ paragraphRange: NSRange, separatorGlyphRange paragraphSeparatorRange: NSRange) {
           super.setParagraphGlyphRange(paragraphRange, separatorGlyphRange: paragraphSeparatorRange)
   
           guard PreferencesManager.shared.shouldShowInvisibles == true else { return }
   
           if let substring = (self.layoutManager?.textStorage?.string as NSString?)?.substring(with: paragraphRange) {
               let expression = try? NSRegularExpression.init(pattern: "\\s", options: NSRegularExpression.Options.useUnicodeWordBoundaries)
               let sunstringRange = NSRange(location: 0, length: substring.characters.count)
   
               if let matches = expression?.matches(in: substring, options: NSRegularExpression.MatchingOptions.withoutAnchoringBounds, range: sunstringRange) {
                   for match in matches {
                       let globalSubRange = NSRange(location: paragraphRange.location + match.range.location, length: 1)
                       self.layoutManager?.addTemporaryAttribute(NSForegroundColorAttributeName, value: Color.invisibleText, forCharacterRange: globalSubRange)
                   }
               }
           }
       }
   }
   

3. To show/hide invisible characters, just call:

   let storageRange = NSRange(location: 0, length: currentTextStorage.length)
   layoutManager.invalidateGlyphs(forCharacterRange: storageRange, changeInLength: 0, actualCharacterRange: nil)
   layoutManager.ensureGlyphs(forGlyphRange: storageRange)