Problem
Manual credit score analysis is slow and depends on many financial, behavioral and profile variables.
technical case study
AI • Machine learning / Classification
Machine learning proof of concept for classifying credit scores from financial and behavioral data.
50,000 monthly records from 6,250 customers (synthetic data): profile, income, debt, delays, loans and score.
With train and test split by customer, Random Forest scores 64.0% and KNN 54.9%. A row-level split pushes it to 80.3%, but that number is inflated.
Confusion matrix on the test set and the features that weigh most in the decision, such as total debt, credit history and credit mix.
The final model, trained on all customers, predicts the score (Good, Standard or Poor) of new customers.
Manual credit score analysis is slow and depends on many financial, behavioral and profile variables.
A machine learning notebook was built to prepare the data, encode categorical variables, train models and compare accuracy between Random Forest and KNN.
With train and test split by customer, Random Forest reached 64.0% on the test set versus 54.9% for KNN. A row-level split raises these to 80.3% and 76.0%, but they are inflated because the same customer appears on both sides.
POC completed, suitable for technical demos and future work on pipelines, validation and explainability.
The project was created to test a complete classification workflow: import a customer dataset, turn text variables into numbers, split train and test, train two algorithms and apply the best model to new customers.
Notebook with tables, model accuracy, confusion matrix, feature importance and the final prediction for new customers.
Python workflow with Pandas and Scikit-learn: Pipelines with OrdinalEncoder (Random Forest) and OneHotEncoder + StandardScaler (KNN), GroupShuffleSplit to split train and test by customer, accuracy_score and classification_report. The best model is saved with joblib.
CSV with 50,000 monthly records from 6,250 customers, generated from the statistics of a real dataset (synthetic data), with occupation, annual salary, accounts, cards, loans, delays, credit history, payment behavior and score.
The project is treated as a technical proof of concept, not as an autonomous tool for approving or rejecting credit.
Because it deals with credit, a real version would require bias analysis, model explainability and auditable criteria.
Each customer has several months of records. The split is done by customer, so the same customer is never in both train and test, and accuracy measures performance on customers the model has never seen.
Loading the customer dataset and a first inspection of the columns.
Converting occupations, credit mix and payment behavior into numeric values.
Separating input and output variables, with a 70/30 train/test split by customer.
Training Random Forest and KNN to compare classification approaches.
Applying the chosen model to new customers to predict their score class.
def criar_modelos() -> dict[str, Pipeline]:
#Monta os dois modelos como pipelines
#Entao a conversão aprendida no treino e exatamente a mesma usada nos novos clientes(nao precisa refazer)
#Random Forest: arvores nao precisam de escala, basta numerar as categorias
preparo_floresta = ColumnTransformer(
[
(
"categoricas",
OrdinalEncoder(handle_unknown="use_encoded_value", unknown_value=-1),
COLUNAS_CATEGORICAS,
)
],
remainder="passthrough",
verbose_feature_names_out=False,
)
#KNN: mede as distancias, entao os numeros precisam estar na mesma escala
#(senao o salario domina tudo) e as categorias viram colunas 0/1
preparo_knn = ColumnTransformer(
[
("categoricas", OneHotEncoder(handle_unknown="ignore"), COLUNAS_CATEGORICAS),
("numericas", StandardScaler(), make_column_selector(dtype_include="number")),
]
)
return {
"Random Forest": Pipeline(
[
("preparo", preparo_floresta),
("modelo", RandomForestClassifier(random_state=SEMENTE, n_jobs=-1)),
]
),
"KNN": Pipeline(
[
("preparo", preparo_knn),
("modelo", KNeighborsClassifier()),
]
),
}
def dividir_por_cliente(x, y, grupos):
#Divide em treino e teste sem repetir clientes entre os dois
#Cada cliente aparece em varios meses. Se a divisao for por linha, o mesmo cliente fica no treino e no teste
#E a IA "aprende" o cliente em vez de aprender o padrao
divisor = GroupShuffleSplit(n_splits=1, test_size=PROPORCAO_TESTE, random_state=SEMENTE)
idx_treino, idx_teste = next(divisor.split(x, y, groups=grupos))
return x.iloc[idx_treino], x.iloc[idx_teste], y.iloc[idx_treino], y.iloc[idx_teste]
tabela = carregar_dados(args.dados)
x, y, grupos = separar_x_y(tabela)
x_treino, x_teste, y_treino, y_teste = dividir_por_cliente(x, y, grupos)
print(f"{len(tabela)} linhas de {grupos.nunique()} clientes")
print(f"Treino: {len(x_treino)} linhas | Teste: {len(x_teste)} linhas\n")
modelos = criar_modelos()
acuracias = {}
for nome, modelo in modelos.items():
modelo.fit(x_treino, y_treino)
acuracias[nome] = accuracy_score(y_teste, modelo.predict(x_teste))
print(f"{nome}: {acuracias[nome]:.1%} de acerto")
melhor = max(acuracias, key=acuracias.get)
print(f"\nMelhor modelo: {melhor}\n")
print(classification_report(y_teste, modelos[melhor].predict(x_teste)))
# Depois de escolher, vc pode treinar o melhor modelo com todos os dados antes de salvar
modelo_final = criar_modelos()[melhor]
modelo_final.fit(x, y)
args.saida.parent.mkdir(parents=True, exist_ok=True)
joblib.dump(modelo_final, args.saida)
args.saida.with_name(ARQUIVO_METRICAS.name).write_text(
json.dumps({"melhor_modelo": melhor, "acuracia_teste": acuracias}, indent=2, ensure_ascii=False),
encoding="utf-8",
)
print(f"Modelo salvo em {args.saida}")
def prever(modelo, novos_clientes: pd.DataFrame) -> pd.Series:
colunas_esperadas = list(modelo.feature_names_in_)
faltando = set(colunas_esperadas) - set(novos_clientes.columns)
if faltando:
raise ValueError(f"Colunas faltando no arquivo de novos clientes: {sorted(faltando)}")
x = novos_clientes[colunas_esperadas]
return pd.Series(modelo.predict(x), index=novos_clientes.index, name="score_previsto")
def main():
parser = argparse.ArgumentParser(description="Prevê o score de crédito de novos clientes.")
parser.add_argument("arquivo", nargs="?", type=Path, default=ARQUIVO_NOVOS_CLIENTES,
help="CSV com os novos clientes")
parser.add_argument("--modelo", type=Path, default=ARQUIVO_MODELO, help="Modelo salvo por src.treinar")
parser.add_argument("--saida", type=Path, help="CSV para salvar as previsões (opcional)")
args = parser.parse_args()
if not args.modelo.exists():
raise SystemExit(f"Modelo não encontrado em {args.modelo}. Rode antes: python -m src.treinar")
modelo = joblib.load(args.modelo)
novos = pd.read_csv(args.arquivo).drop(columns=[COLUNA_ID, COLUNA_ALVO], errors="ignore")
novos["score_previsto"] = prever(modelo, novos)
print(novos[["idade", "profissao", "salario_anual", "score_previsto"]].to_string())
With the split by customer, Random Forest reached 64.0% test accuracy and KNN 54.9%. Comparing it with the row-level split (80.3%) showed how inflated the evaluation gets when the same customer appears in both train and test.
The project reinforced the importance of preprocessing, model comparison and responsibility when presenting AI applied to credit.