[FIXED] Replace entire line in text file with python regex

Issue

Trying to do something simple... replace a line of text in the MongoDB configuration file. This is the small area of 'mongod.cfg' I'm working with:

# Where and how to store data.
storage:
  dbPath: D:\MongoDB\data
  journal:
    enabled: true
#  engine:
#  wiredTiger:

I want to replace the default " dbPath: D:\MongoDB\data" line with the path I have gotten from user input. This is a code snippet that I found on the internet that does what I want... but for some reason (I think!) it's choking on the regex.

def replace(filePath, text, subs, flags=0):
   with open(filepath, "r+") as file:
       contents = file.read()
       text_pattern = re.compile(re.escape(text), flags)
       contents = text_pattern.sub(subs, contents)
       file.seek(0)
       file.truncate()
       file.write(contents)
       

filepath=mongocfgpath
text="^.*dbPath:"  # also tried "^\s*dbPath:"
subs="  dbPath: " + dbpath
#calling the replace method
replace(filepath, text, subs)
print(subs)

This is the input and the error:

Please enter the path of the installed MongoDB: E:\pydev\mongocryptsetup310
Please enter MongoDB data directory path: e:\db

Traceback (most recent call last):
  File "D:\Python310\lib\sre_parse.py", line 1039, in parse_template
    this = chr(ESCAPES[this][1])
KeyError: '\\d'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "e:\pydev\mongocryptsetup310\mongocrypt.py", line 52, in <module>
    replace(filepath, text, subs)
  File "e:\pydev\mongocryptsetup310\mongocrypt.py", line 43, in replace
    contents = text_pattern.sub(subs, contents)
  File "D:\Python310\lib\re.py", line 326, in _subx
    template = _compile_repl(template, pattern)
  File "D:\Python310\lib\re.py", line 317, in _compile_repl
    return sre_parse.parse_template(repl, pattern)
  File "D:\Python310\lib\sre_parse.py", line 1042, in parse_template
    raise s.error('bad escape %s' % this, len(this))
re.error: bad escape \d at position 12

I'm sure I'm just doing something stupid. :) This is Python 3.10.2 running on Windows. I'm importing 're' earlier in the code, not displayed.

Solution

The problem is that you are passing e:\db as a part of subs variable and it has a backslash resulting in a special character. So before using it for replacement all backslashes have to be replaced, i.e.:

contents = text_pattern.sub(subs.replace('\\', r'\\'), contents)

On the other hand the input (text) has to be escaped only partially (not it's regexp part).

Besides that, regexp in your case has to be used with re.MULTILINE (8) flag because you are replacing something in the middle of a file with many new lines.

Anyways something like this will do:

import re

def replace(file_path, new_db_path):
    with open(file_path, "r+") as file:
        contents = re.sub(r"(^\s+dbPath: ).+", rf"\1{new_db_path.replace('\\', r'\\')}", file.read(), flags=re.MULTILINE)
        file.seek(0)
        file.truncate()
        file.write(contents)

replace("mongod.cfg", r'e:\db')

P.S: replace() function from your example has a case-typo and thus uses undeclared variable filepath

Answered By - Programmierus