[FIXED] Can't get the correct paths of files inside a hidden directory

Issue

I'm trying to get the absolute paths of files in a hidden directory using a for loop:

import os
from pathlib import Path

dir = "/home/user/myproject/.hidden_dir/last_dir"
for files in list(filter(os.path.isfile, os.listdir(dir))): # to make sure we ignore directories
    print(Path(files).absolute())

So it's giving me off this result:

/home/user/myproject/project.py
/home/user/myproject/todo.org

It looks like it iterates through the right directory because those files aren't located in /home/user/myproject but in /home/user/myproject/.hidden_dir/last_dir in fact I think it's just a matter of truncated results, as if it didn't want to show the hidden directory and it's children. I tried to replace print(Path(files).absolute()) with print(os.path.abspath(dir)) but it's the same result again.

But then if I try:

print(Path(dir).absolute())

I get the correct path including the hidden directory:

/home/user/myproject/.hidden_dir/last_dir

I can't wrap my head around this.

Solution

This actually has nothing to do with hidden directories, but rather with a confusion with how os.listdir works. If you look at the results of os.listdir(dir), you'll see that it returns the base file name of the files in the directory. A file with the path /home/user/myproject/.hidden_dir/last_dir/project.py will be represented as "project.py" in the result. The Path() object doesn't check if the path you pass in actually exists, and if it's not an absolute path, it will assume it's under the current working directory. In this case, it seems like the working directory was /home/user/myproject/, giving you the weird results you saw.

The right way to do something like this would be

dir_path = Path("/home/user/myproject/.hidden_dir/last_dir")
for file_path in filter(lambda p: p.is_file(), dir_path.iterdir()):
    print(file_path.absolute())

Answered By - rchome